The "Automotive AI Agent Product Development and Commercialization Research Report, 2024" report has been added to ResearchAndMarkets.com's offering.

Automotive AI Agent product development: How to enable "cockpit endorser" via foundation models?

According to OPEN AI's taxonomy of AI (a total of 5 levels), AI Agent is at L3 in the AI development path

Limited by interaction modes and tool usage capabilities, popular foundation models in 2023 can only reach L2 (Reasoners) at most. In contrast, developing the automotive AI system by building automotive agents is a more appropriate goal: Agent improves weak links of application of foundation models in scenarios by way of calling active intelligent features and multiple tools/foundation models, further improving cockpit intelligence level.

Agent is the endorser of emotional cockpits

'Emotional cockpit' has been nothing new for multiple years, but actually realizing it still starts with the introduction of foundation models in vehicles. Under specific triggering conditions, voice assistant chats with the user through preset emotional corpus, but it cannot adapt to human dialogue logic in real chat scenarios. After being applied to vehicles, Agent integrated with multiple foundation model bases can recognize the environment more accurately, and more tool library interfaces further enhances its generalization capability to cope with chat and Q&A in diversified scenarios, truly realizing the warm companionship of the 'cockpit endorser'.

The design of mainstream emotional interaction scenarios focuses on emotion recognition, user memory, and behavior arrangement. Some OEMs and Tier1s have also launched technologies or products to enhance the emotional value of Agents:

Sore points that need to be solved to improve user experience

Imagine that an intelligent cockpit can not only understand and execute instructions given by the car owner, but also predict the owner's needs, just like a thoughtful personal assistant. Will this make car owners more excited? Compared to buying a traditional car and having to explore each function on one's own, everyone wants a cockpit 'endorser' which can help manage all cockpit functions as they just say a few words. Agent is a time-saving and trouble-free solution.

Currently most Agents introduced in vehicles still serve as an assistant and a companion listing functions for specific scenarios. Yet compared with foundation models, Agents feature greater potential, motivated autonomy, and outstanding tool-using capabilities, more fit with the label of 'active intelligence', and can even make up for the limitations of foundation models in practical applications.

There is however still a long way to go in technology development to make automotive agents truly 'active and intelligent' and meet users' experience value. Agent needs to be more precise in active perception, data processing, state recognition, etc., accurately understand the environment, judge real needs of people in the car, and then adopt corresponding strategies. Wherein, one of challenges lies in Agent's accurate judgment of user needs. Compared with passive interaction in normal circumstances, active intention recognition lacks voice commands. In the process of environment/personnel/vehicle state recognition, it may not be possible to obtain a description that is extremely close to the current scenario through vector feature matching, and the preset solution may not satisfy the real intentions of people in the car.

At present, most recommended functions are just to execute preset instructions. This limits' active and intelligent' capabilities of Agent and leads to frequent sore points in the reasoning process. For example, if Agent fails to accurately understand the current scenario, it may not make recommendations as expected, for instance, recommending music or navigation at a wrong time. The final result is to affect user experience and make the Agent become a 'guessing machine' to users.

In addition, Agent also has shortcomings in perception when receiving voice commands. According to the publisher's incomplete statistics on sore points in automotive agent use cases of some car owners, the most frequent sore points are wake-up failure, recognition error, and false wake-up.

Among the 120 cases, wake-up failure, recognition error, and false wake-up are mentioned 19, 18, and 17 times respectively, namely, accounting for 16%, 15%, and 14%. Other sore points also include unavailability of see-and-speak, semantic clarification and continuous commands, inability to recognize dialects, and delayed response, totaling 89 sore points in voice link, or 74.2% of the total in this statistical survey.

Furthermore, a range of problems caused by unreasonable Agent architecture/scenario design also include irrational scenario triggering conditions, secondary wake-up of foundation models, failure of long/short-term memory, and recommended actions made autonomously according to owners' habits but failing to meet expectations, which respectively reflect limitations of Agent in scenario setting, architecture deployment, memory module, and reflection module.

Key Topics Covered:

Overview of Automotive AI Agent

• Definition of Agent
• Development History of Agent
• Foundation Models Regain Vitality Using the Agent Concept
• Differences between Foundation Models, Agents, and AIGC
• Automotive AI Agent Product Definition
• Automotive AI Agent based on Multi-agent System: Module Design
• Automotive AI Agent based on Multi-agent System: Component Functions
• Automotive AI Agent based on Multi-agent System: Component Characteristics
• Automotive AI Agent Reference Architecture (by Functional Module and Component)
• Automotive AI Agent Reference Architecture (by Deployment Level)
• Original Diagram of NIO (Nomi) Architecture
• Original Diagram of NIO (Nomi) Deployment
• NIO (Nomi) Module Design
• NIO (Nomi) Module Design - Multimodal Perception
• NIO (Nomi) Module Design - Command Distribution
• NIO (Nomi) Module Design - Scenario Customization and Creation Process
• Highlights of NIO (Nomi)
• Original Diagram of Li Auto (Lixiang Tongxue) Architecture
• Li Auto (Lixiang Tongxue) Module Design
• Li Auto (Lixiang Tongxue) Supporting Facilities - Data/Training Platform
• Li Auto (Lixiang Tongxue) Supporting Facilities - Reasoning Engine
• Original Diagram of Xiaomi (Xiaoai Tongxue) Architecture
• Xiaomi (Xiaoai Tongxue) Module Design
• Zeekr Agent Module Design
• Original Diagram of Neta Agent Architecture Deployment
• Neta Agent Module Design
• Original Diagram of BAIC Agent Architecture Deployment
• BAIC Agent Module Design
• Huawei (Pangu Agent) Module Design
• Original Diagram of AISpeech Agent Architecture Deployment
• AISpeech Agent Module Design
• Original Diagram of Lenovo Agent Architecture Deployment
• Original Diagram of Zhipu Agent Architecture Deployment
• Zhipu Agent Module Design
• Original Diagram of Tinnove Agent Architecture Deployment
• Tinnove Agent Module Design
• Agent Architecture Design Process: Framework Selection
• Agent Architecture Design Process: Tool Calling Method
• Comparison of Automotive AI Agent Architecture

Key Issues in Development of Automotive AI Agent Products - User Sore Points and Technical Difficulties

• Classification of Automotive AI Agent Scenario: Typical Commands in Different Scenarios
• Classification of Automotive AI Agent Scenario: Case (1) NIO
• Classification of Automotive AI Agent Scenario: Case (2) Li Auto
• Classification of Automotive AI Agent Scenario: Case (3) Xiaomi
• Automotive AI Agent Scenario Design Case: Q&A Scenario
• Automotive AI Agent Scenario Design Case: Mobility Scenario
• Automotive AI Agent Scenario Design Case: Chat Scenario
• Automotive AI Agent Scenario Design Case: Q&A/Office Scenario
• User Sore Points in Different Agent Usage Scenarios: Summary
• User Sore Points (1): Vehicle Control Scenario
• User Sore Points (2): Mobility Scenario
• User Sore Points (3): Q&A Scenario
• User Sore Points (4): Entertainment Scenario
• Agent Technical Difficulties
• Agent Technology Case: Intent Recognition
• Agent Technology Case: Reasoning Acceleration
• Agent Technology Case: Streaming Voice
• Agent Technology Case: Emotional Interaction
• Agent Technology Trends: Two Keys to Achieving Active Intelligence
• Agent Technology Trends: Two Mainstream Design Methods for Emotional Anthropomorphism

OEMs' AI Agent Investment, Development, and Operation

• Comparison of Automotive AI Agent Development Support
• OEMs' Planning for Automotive AI Agents
• Comparison between Three Automotive AI Agent Development Modes: Advantages/Disadvantages
• Comparison between Three Automotive AI Agent Development Modes: Cost
• Position Setting of OEMs' AI Agent Team
• Case of OEMs' AI Agent Team Position Setting
• Positions Recruited by Chery AI Agent Team
• Positions Recruited by Geely AI Agent Team
• Positions Recruited by Li Auto AI Agent Team
• Positions Recruited by NIO AI Agent Team
• Positions Recruited by Xiaomi AI Agent TeamAI Agent Development Cycle and Operation Mode
• AI Agent Business: OEMs' Profit Model
• AI Agent Business: Suppliers' Profit Model
• AI Agent Business: Suppliers' Charging Standards
• Commercial Development Trends of Automotive AI Agents

Automotive AI Agent Suppliers and Their Supply Relationships

• Cockpit Base Foundation Model: Model Configurations
• Cockpit Base Foundation Model: Selection Reference Factors
• Cockpit Base Foundation Model Suppliers
• Industry Chain of Vector Database Suppliers
• Comparison between Vector Database Products: Chinese Vector Databases
• Comparison between Vector Database Products: Foreign Vector Databases
• Vector Database Supplier Cases
• Comparison between Voice ASR Module Suppliers
• ASR Module Supplier Cases
• Cockpit Data Collection Sensors: Mainstream Configurations/Data Collection Regulations
• Sensor Data Processing Cases

Companies Featured

• Chery
• Geely
• Li Auto
• NIO
• Xiaomi
• Zeekr
• Neta
• BAIC
• Huawei
• AISpeech
• Zhipu
• Tinnove
• Lenovo

For more information about this report visit https://www.researchandmarkets.com/r/8nhq7g

About ResearchAndMarkets.com

ResearchAndMarkets.com is the world's leading source for international market research reports and market data. We provide you with the latest data on international and regional markets, key industries, the top companies, new products and the latest trends.

View source version on businesswire.com: https://www.businesswire.com/news/home/20250122978068/en/