When ML meets Product: October ’24 AI Product Updates
Keeping up to date with new AI models, products, use cases, trends, and resources
Welcome to the October edition of “When ML Meets Product — AI Product Updates”. As the Artificial Intelligence field rapidly advances, staying up-to-date with the latest developments and understanding their impact on business and products has never been more essential. This edition brings you October’s top updates:
- GenAI Model Updates: Latest breakthroughs from major foundational model players.
- AI Product Updates: Product trends and cool new products that leverage AI.
- Other Relevant Resources: Stay informed with additional resources.
GenAI Model Updates
The competition among GenAI model providers (still) continues. Over the past month:
- OpenAI released real-time API (enabling instant voice interactions and conversational AI experiences), prompt caching (reducing costs for repeated prompts from your system), canvas (a new interface, similar to Anthropic’s artifact, that allows better interaction and iteration on ChatGPT’s outputs). As many people expected, they launched ChatGPT search, moving one step forward to generating answers by searching the web and linking the relevant web results (and potentially becoming a Google competitor!).
- Anthropic surprised the world this month with several demos on how their new product could control your PC. Demos included creating and running a website from 0, or filling out forms with the relevant information automatically. In the demos you can see how the agentic system plans the necessary steps to achieve a given task, and runs steps like tacking screenshots, moving the mouse, clicking and typing.
- Meta released Llama 3.2, including multilingual text-only models (1B, 3B) and text-image models (11B, 90B), with quantized versions of 1B and 3B to fit onto edge and mobile devices. After Google’s NotebookML success (generate a podcast from any file), they will also be releasing the open source version of it, NotebookLlama.
- Nvidia (chip manufacturer) entered the open-source space with its NVLM 1.0 family of large multimodal language models.
AI Product Updates
AI continues to drive significant advancements across various sectors, with marketing and advertising standing out as early adopters and major beneficiaries. Following this wave, Amazon launches new tools for their advertisers, with ads and audio creation features to empower advertisers to create campaigns more efficiently.
AI is also transforming e-commerce, we’ve seen a lot of innovation especially in the product publication workflow, as well as optimizing search, discovery, and findability of products. Another example from Amazon in this area: now launching a title and description personalization based on a customer’s shopping activity, with an aim to improve relevancy and customer engagement.
Another sector experiencing substantial benefits from GenAI is software development. Recently, Google CEO Sundar Pichai announced that 25% of all new code at Google is now generated by AI, yet another example on how GenAI is already boosting developer productivity and accelerating code generation.
More on the enhanced work productivity topic: Glean, a startup recently name by OpenAI as one of their top 5 competitors (alongside with Anthropic, xAI, SafeSuperintelligence, and Perplexity), is a promising company focused on making it easy to access and collaborate with all relevant information and knowledge from an enterprise. Glean connects with all data sources from a company, allowing employees to easy find, understand, and conversate with the information they need. This can boost productivity by supporting tasks like content creation, summarization, and task automation. I recently spoke with someone whose company uses Glean, and they had great feedback about it!
Forbes’ Real-World Uses of AI in Business:
- Tech, telecommunications, automotive, and financial services sectors are leading AI adoption, particularly in areas like industrial machine learning, cybersecurity, and robotics.
- There is a growing number of AI-first startups, which leverage AI to implement products with agility and innovation, focusing in niche solutions.
- Key applications include chatbots and virtual assistants, personalization engines, and AI-powered sales and marketing assistants.
Other relevant resources
There are raising controversies around the benchmarks and test sets used to evaluate and compare LLMs, and quantify their “intelligence”. A primary concern is that some of these test sets may have been leaked into the training data, compromising the validity of these benchmarks. Recent research from Apple checked one popular test set, GSM8K dataset (Grade School Math 8K), consisting of over 8k grade school level math problems. The research concluded that LLMs performance dropped when models were exposed to similar types of problems that contained small variations or extra irrelevant information. For more on this topic, see the latest isse of The Batch latest issue — Benchmarks Are Meaningless.
Great overview of LLM evaluation metrics: differentiating between statistical and model based scorers, and to cover measures like answer relevancy, correctness, hallucinations, contextual relevancy, responsibility, task-specific.
New State of AI report: frontier lab performance begins to converge, research focuses on planning and reasoning in LLMs, multimodality is everywhere, $9 trillion enterprise value for AI firms, concerns about existencial risks have cooled down, and more!
How AI affects our sense of self, with key insights such as:
- Recieving good news about decions and evaluations are better percieved if it is a human who delivers it than if it is an AI or an automation. However, this preference disappears when recieving bad news.
- For products with symbolic or emotional value, people tend to value them more when they are made or designed by humans rather than AI. However, this preference is less disappears for more functional or utilitarian products.
- People need to feel a sense of proud and involvement, rather than feeling replaced by AI.
We Need to Talk about AI Branding, by Kira Klaas. Great discussion on how and if to brand your AI features (do you need to highlight AI in your product?), using AI to make your user look good, and focusing on utility over technology.
Inbal Shani on maturing your product leadership in the age of AI. Great discussion about how AI is reshaping product leadership, emphasizing customer-centered design and cross-functional collaboration. Highlights that success with AI requires holistic, critical thinking and adaptation to customer needs, while being aware of risks such as data privacy and security.
Lessons in product leadership an AI strategy from Glean, by Tamar Yehoshua. On top of discussing Glean (enterprise productivity product mentioned above) and the impact of AI on product management and the future of work, Tamar shares incredible strategic and business insights together with great career advice and learnings.
Wrapping it up
That was it from When ML meets Product — October’24 AI Product Updates. Hope you enjoyed the read!
I’ll be happy to hear your thoughts, questions, or suggestions.
And if you want more content, check out my latest blogpost AI feels easier than ever before but is it really? It covers what I consider to be today’s four major challenges that make building AI products complicated: prediction feasibility, cost of being wrong, societal impact, and production reality.
