Artificial intelligence is already a big buzzword for 2024, and it’s only a few weeks in. Just about everything worth talking about at the Consumer Electronics Show had some sort of AI story, and we’ve just seen the back of Samsung’s UnPacked event, where its new Galaxy S24 phones have ‘Galaxy AI’ at their core.
I had a chance to visit Samsung’s Research America R&D facility in the heart of Silicon Valley to go deeper into the new AI-enhanced Galaxy S24, not to mention what makes it ‘smart’ for phones going forward.
What is an AI on phone anyway?
‘AI’ can mean a lot of things, especially when marketers start sharpening their pencils. Sometimes, an ‘AI’ logo stamped on a device just means that a complex algorithm has been ‘baked-in’ allowing it to do something better. This algorithm is probably pre-formulated using a large server farm somewhere, so there’s no ‘machine learning’ conducted on the device itself.
An example could be an electric toothbrush that’s marketed as being able to teach you how to brush better using AI. It leverages a large database of people’s brushing habits, and an AI learning model, but none of this happens on the toothbrush itself. This just compares the pre-computed results to what it learns about your brushing to give you the most appropriate tips. It may even occasional refresh its algorithm via an online update. So you could say that this is AI but the device you’re using isn’t really capable of it.
On-device AI
The Samsung Galaxy S24 operates differently. Its AI features are made possible in two ways, via actual AI processing ‘on-device’ as well as the ‘in-the-cloud’ type of AI. On-device AI happens in real-time using the phone’s processing power. An example could be speech-to-text-translation, where a phone ‘hears’ and ‘understands’ your speech and then transforms it into text.
This requires a powerful Neural Processing Unit (NPU), which is part of the S24 Ultra’s exclusive Snapdragon 8 Gen 3 chipset. Also needed is a training model for the system to understand how to turn sounds into text. In the ‘old days’, this type of task would be impossible on a smartphone due to hardware limitations.
In addition to advances in processing power, the Galaxy S24 employs a variety of compression optimisations including 3-bit quantisation and techniques like training for multiple languages rather than just one, to create more on-device AI capability than ever before.
As a result, the S24 can tackle a variety of tasks including Speech Recognition (voice to text), Neural Machine Translation (translating one language to another), and Voice Synthesis (converting text to natural voice).
Another considerable benefit of ‘on-device’ AI is that people’s personal information doesn’t need to be sent to the internet, so it stays safe. In addition to privacy, on-device AI processing is much faster than waiting for information to transfer to the cloud to processed by a neural network, and then sent back again.
In-the-cloud AI
While the Galaxy S24 Ultra has plenty of on-device power, it still needs to rely on off-site processing for some services. For the most part, this includes ‘generative’ tasks, such as re-drawing and touching-up photos. For example, generative AI is needed to extend the background of a lopsided photo to straighten it up, or to remove an unsightly object such as a lamp post from an urban landscape.
Also, Samsung is collaborating with Google to leverage its latest AI, called Gemini. This multi-modal AI drives the new ‘Circle to Search’ feature, and if you haven’t tried it yet, it looks to be a game-changer. You can circle, highlight, squiggle or press nearly anything on your screen to find out more about it using Google Search. You could be watching a video and spot a great hat that you want to buy or highlight some text in a message about a town that you want to visit. The genius here is that the AI can identify a picture, text, concept or what have you, and attach context and meaning to it.
Gemini ‘Pro’ will also assist with Samsung’s Notes, Voice Recorder and Keyboard apps, delivering “better summarization features”, according to Google.
Samsung has also developed its own ChatGPT-style AI technology too, and this is called GAUSS. This Large Language Model (LLM) helps with language-focused tasks such as summarising written and spoken information, even across different languages. For example, it can create a transcript of a meeting with different language speakers, and then translate what is being said.
Overcoming AI’s challenges
According to YJ Kim, Samsung’s Executive Vice President and the Head of Artificial Intelligence Team for Samsung Mobile eXperience Business, the biggest challenges they faced bringing AI features to the Galaxy S24 were size and performance. Size, in terms of the learning model, and performance by way of running an on-device neural network. However, with the power delivered through the specialised Snapdragon 8 Gen 3 and Neural Processing Unit, as well as optimisations in Quantisation and compression, they made it work.
Kim also said that the Live Translate feature was especially challenging because existing models were trained on written language, not spoken, so a lot of work needed to be done to optimise for verbal elements.
Who pays for AI?
With AI becoming an essential feature-set for the latest generation of premium phones, providing these services is not cheap. Off-site processing, for example, requires servers and storage, plus there are fees for using partner Ais such as Google’s Gemini.
The net result is that Samsung has stated that its AI services will only be available for free until the end of 2025. After this point Samsung may need to charge for them, probably in the form of a subscription.
So, even if you’ve bought a Galaxy S24 Ultra, which comes with AI features today, you may not have access to them permanently. So the future costs of phones may be less about the hardware and more about the subscription fees for AI functionality that we can’t live without!
Is AI trustworthy?
When it comes trust, there have been some instances where bias and bigotry have found their way into AI-generated results. During my visit to Samsung’s R&D centre, Samsung’s YJ Kim, Executive Vice President and the Head of Artificial Intelligence Team for Samsung Mobile eXperience Business, said that there are filters in place to prevent unacceptable responses, so it looks like we’re safe for now. Also, as the current AI services are generally for translating, summarising, searching and photo editing, there are less opportunities for these issues to creep in, as opposed to a full LLM like ChatGPT.
What does the future hold?
While an all-knowing smartphone is still a ways off, the next major development we can expect from AI is greater personalisation. For example, the Live Translate feature can convert my English voice into a synthesised French voice, but doesn’t sound like me. Soon, we can expect Live Translate to convert my voice to a French version that still sounds my voice, and not a synthesised one.
I also asked YJ Kim why we haven’t seen Bixby (Samsung’s version of ‘Google Assistant’) get smarter. I would have thought that Bixby could become a verbral version of a ChatGPT-style LLM, so you could, in essence, ask it anything and it would give you a natural sounding and intelligent reply. Imagine using a phone that would know what information you want before you asked it, find out anything, make calls for you, summarise your mail, feeds and news and even write you a love poem. This is what a LLM running on your phone could do.
Unfortunately, we aren’t going to be seeing this any time soon because the on-device processing demands are just too high, and LLMs are much too large to squeeze onto a phone. It will be interesting when this happens however, as then, a phone might truly become your thinking, pocket-sized personal assistant.
So, I’m calling it now – with Galaxy AI, phones are changing forever. They’re not smartphones anymore, they’re ‘thinkphones’. Don’t forget that you heard it here first!
Valens Quinn attended Samsung’s Research America facility in Mountain View CA as a guest of Samsung Australia
The post Samsung’s Galaxy S24: Rise of the ‘Thinkphone’ appeared first on GadgetGuy.
0 comments:
Post a Comment