The news

On July 17, 2025 Zoho launched Zia LLM and deepened its AI portfolio with agents, an agent builder, MCP support and an agent marketplace.

Key announcements from the press release include:

• In-House LLM: Zoho has developed its own large language model, Zia LLM, which comes in three sizes (1.3B, 2.6B, and 7B parameters) to optimize for different business use cases. This allows customers to leverage AI while keeping their data within Zoho's ecosystem, ensuring privacy. The three models allow Zoho to always optimize the right model for the right user context, striking the proper balance between power and resource management. This focus on right-sizing the model is an ongoing development strategy for Zoho.
• Speech-to-Text Models: The company also unveiled two proprietary Automatic Speech Recognition (ASR) models for English and Hindi, with plans to support more languages in the future.
• Prebuilt AI Agents: To facilitate immediate adoption, Zoho has introduced a range of AI agents that are integrated directly into its products. These agents are designed to automate tasks for various business roles such as sales development, customer support, and account management.
• Global and Private Cloud Deployment: The new Zia LLM will be deployed across Zoho's data centers in the US, India, and Europe.
• Continued Support for Other Models: While promoting its own AI, Zoho will continue to support integrations with other popular large language models like ChatGPT, Llama, and DeepSeek.
• Zoho will continue to scale Zia LLM’s mode sizes. A2A capabilities are on the roadmap.

The bigger picture

Enterprise software has been a platform game for a long time. AI, in particular generative and agentic AI, have upped the ante in this respect. The competition has become even more one that is based around platforms. An enterprise software vendor that wants to address a significant portion of an enterprise’s value chain cannot go without agents, an agent builder, and a marketplace anymore. This is true whether they use an own AI, a derivative of an open source one, or build it using their own stack.

Plus, support of MCP becomes non-negotiable as most customers run on more than one platform. A2A support then is the next step. Both can also be used as a bridge head and Trojan horse to work towards an increased share of the customers’ value chains.

At the same time this AI battleground is evolving with an enormous pace which seems to accelerate every day. This makes it important for buyers to have a deep look at who to engage with while vendors need to offer a strong narrative around a compelling value proposition.

My point of view and analysis

With this announcement it is clear that Zoho is serious about AI. With this, the company raises its AI profile significantly, particularly in the CX space, as Zoho CRM is its most mature product.

The company works on AI capabilities, tooling and infrastructure tor quite some time now. Plus, it owns its hardware and data centers. This includes already having deployed more than 80 AI algorithms in the past.

Now it also has a family of LLMs – a family that is poised to grow, initially with a 32bn parameter model. And frankly, to me this is the biggest splash although all other parts of the announcement show a round offering.

One of the core questions that I asked myself is whether the world needs another LLM or whether it would be sufficient to adapt an existing open source one to own needs. On the other hand, this would not be the Zoho way. Zoho is adamant in staying in control of its own destiny. As part of this, the company is convinced that it needs to own its software stack, as only this way gives full control, full understanding of its behavior and all options. This is certainly also true for an LLM. As Raju Vegesna, Chief Evangelist of Zoho, said: “If AI is going to be the future, and if you are a technology company, what’s more important than creating that core part of technology that is going to define its future?”

In addition, Zoho’s LLM family comes with a twist. It is not addressing consumer tasks but is specialized on what Zoho customers require. And Zoho customers are predominantly B2B businesses. In that sense, and true to the objective of delivering “contextual AI”, Zoho follows a route that is similar to the one that SAP takes with its own foundation model. In contrast to SAP, Zoho did not train the model on customer data (SAP trains its foundation model with customer consents) which is in line with the company’s strong stance on privacy and trust. Offering differently sized models is all about “rightsizing”, i.e., the economic and performant use of valuable compute resources. In essence, Zoho offers to not bring a sledgehammer to a job when a mallet is sufficient. If the system is able to select the right model automatically, this can be a game changer.

The second really interesting part of the announcement is about its automatic speech recognition (ASR) models. This is again a matter of being independent. Further, a well-functioning ASR can help businesses raise significant efficiencies, if the models work well. As per Zoho’s tests they compare well against larger models. The current limitation is that the ASR models support only English and Hindi at the moment. I expect further widely used languages being supported soon.

The offering of prebuilt agents, an MCP server, an agent studio and a marketplace show that Zoho is serious, but they are also table stakes these days. They are the foundation for scale. These offerings, in combination with the upcoming A2A support, open up the ecosystem game by enabling partners to innovate fast and to facilitate getting into new accounts by being open.

All in all, this is a bold, yet not unexpected move by Zoho. It helps the company to strengthen its enterprise credibility, while maintaining the needs of SMBs squarely in the sights. Especially considering Zoho’s strong stance on value and pricing, the company now places significant pressure on the competition but also on itself, as it is still testing whether it can deliver these capabilities as part of the subscription or whether at least parts of it need to get priced. But just doing this exercise with the goal of delivering these capabilities as part of the package. Even if Zoho will not be able to absorb the cost of running models and agents, pricing can be expected to be extremely competitive.

As Raju Vegesna said, it is a marathon.

And this marathon has only just started.

The analyst’s day is full of research. Now, this is the age of AI and AI is here to help, isn’t it? As everyone is talking about copilots and AI agents, why not using the tools at hand to do a little research on research.

NB., no one really has a good definition of an AI agent, so this might become an additional topic for research.

But I digress.

Imagine the following project at hand, which is not only interesting for analysts, btw, but also for a variety of roles in the corporate world. Let’s call it vendor (competitor) monitoring. The job is the following:

• Research reputable sites for news about a number of vendors, relating to a set of keywords. Reputable sites are high quality news sites, high quality tech publications, high quality analyst sites and, of course the news pages of the vendors in question.
• Limit the time frame of the search matching to the cadence of my information requirement, e.g., “yesterday” for a daily update or “last week” for a weekly update
• Provide a summary of the news
• Give an assessment of how the news affects the positions of the vendors in the marketplace re the key words in question
• Provide these news with their assessments as a prioritized list, sorted from high impact to low impact
• Add an executive summary as a preface
• Send it to me as an email

So far, so simple. After all, a lot of folks, yours truly included, do this every day. And it is taking quite some time. So, this job is a perfect one for an automated update beyond a CSS feed. And it seems like a perfect job for an LLM turned agent – or is it a copilot?

Now, the basic question is: Which one to use? After all, there are plenty, from free to not so free ones. Answering this question turns into yet another interesting experiment: Why not ask some LLMs for their evaluation of suitability? Kinda meta, but an interesting one.

So, I did just that: I asked ChatGPT 4.5, Grok 3, and Gemini in its 4 versions 2.0 Flash Thinking Experimental, 2.0 Flash, 1.5 Pro with Deep Research, 2.0 Pro Experimental for their analysis of which of them is best suited for the research task at hand.

For this, I used the following, simple prompt:

describe the different capabilities and limitations of Gemini 2.0 Flash Thinnking Experimental, 2.0 Flash, 1.5 pro with deep research, 2.0 pro experimental, Grok 3 and chatGPT 4.5, both with and without deep reasoning. Which model is best to support the following use case:   

research the web for news on a given set of companies and a given set of topics. The news shall cover the past 2 days only   

assess the news regarding their impact on the companies' market positions re the given set of topics   

create this in the form of a report   

do this as a daily scheduled task  

accuracy, reasoning and reliability are of high importance. Speed is of lower importance.  

generate a comparison table, give a recommendation and justify the recommendation.

The results are quite interesting.

• Gemini 1.5 Pro with Deep Research resolves to Gemini 2.0 Flash because it offers “a balance of reasoning, accuracy, reliability, and tool use necessary for fulfilling the requirements of the specified use case. Its production-ready status, combined with its ability to handle complex analysis and generate comprehensive reports, makes it the ideal LLM for this task.” It looks at Grok as the runner up.
• Gemini 2.0 Pro Experimental recommends Gemini 1.5 Pro with Deep Research as it “offers the best balance of accuracy, reasoning, and reliability for your demanding research and reporting task, leveraging Google's strengths in both LLMs and web search.”. This is closely followed by Grok 3.
• Gemini 2.0 Flash Thinking Experimental suggests doing a competitive evaluation of ChatGPT 4.5 with Deep Reasoning and Gemini 2.0 Pro Experimental – as they are a tie. “Which is truly "better" in practice will depend on your specific prompts, data, and tolerance for latency and cost”.
• ChatGPT 4.5 (without Deep Research) suggests going for ChatGPT 4.5 with Deep Research as the model “offers enhanced reasoning capabilities, reduced hallucination rates, and a broader knowledge base, aligning well with the requirements for accuracy, reasoning, and reliability in daily scheduled tasks. While models like Gemini 2.0 Flash Thinking Experimental and Grok 3 also provide advanced features, ChatGPT 4.5's maturity and proven track record make it a suitable choice for generating comprehensive and reliable reports.​”.
• Grok 3 in Deep Research mode suggests using Gemini 2.0 Pro Experimental as its “advanced reasoning capabilities, as evidenced by its performance in complex tasks and 2 million token context window, make it ideal for researching news, assessing market impact, and generating daily reports (Gemini 2.0 Pro). The integration with Google Search ensures access to recent news, and as a Google product, it likely offers high reliability for scheduled tasks, aligning with the emphasis on accuracy and reasoning over speed. While Gemini 1.5 Pro with Deep Research is tailored for research, Gemini 2.0 Pro Experimental, being a newer model, likely offers superior capabilities”. Grok looks at Grok as the runner up is it offers advanced reasoning and Deep Search “but potential biases from X data integration”.

So, what does this tell me?

There is probably a bit of self-serving involved in the LLM’s assessments and suggestions. At least Google consistently suggests a Google model and ChatGPT suggests itself. What is a bit confusing is that

An interesting side remark is that only Gemini 1.5 Pro with Deep Research, ChatGPT 4.5 and Grok 3 provide the sources used for the research. Perplexity does this, too. Providing references is important for validating the results.

It looks like the results delivered by these LLMs seem to favor Gemini 2.0 Pro Experimental and ChatGPT 4.5, though, although I am impressed by Grok 3. On the other hand, one needs to know that “experimental” means exactly that – the models are not yet fully stable.

Having said this, if one needs to perform research tasks, as many of us need to, environment matters. Especially smaller businesses often run Google Workspace. In the case that they subscribed to the Business Standard Edition (like I am doing), Gemini is readily available, there is probably no immediate need to purchase an additional ChatGPT license (I have a pro subscription) or a Grok or Perplexity subscription. This is especially true as most of these tools use a lot of the data that users provide to improve their services, which is especially true for free services. Grok, in its privacy statement explicitly recommends to not input any personal data – as it will be used.

In summary, if and when I need to do research, I’ll use Google Gemini 1.5 Pro with Deep Research and Gemini 2 Pro Experimental as my preferred option, simply because ChatGPT 4.5 with Deep Research only offers limited runs per month. As it doesn’t cost much, additionally running the same research – potentially with a slightly changed prompt to cater for model differences – I will use ChatGPT and (if no sensitive data involved) Grok 3 in addition. Worst case, this gives me additional food for thought.

What do you think?

During ZohoDay24, Zoho amongst other topics, gave some insight into how the company looks at AI. Raju Vegesna presented Zoho’s AI vision and progress. Additionally, I had the opportunity for a one on one with Zoho’s director of AI research, Ramprakash (Ram) Ramamoorthy. If you want to listen and watch the interview, you can do this here.

Both represented a vision that is refreshingly differentiated from the current hype with everyone and their dog talking like large language models, LLMs, are the everything one needs.

Well, let me tell you: They aren’t. But let me come to this point later.

In addition to not every language model being created equal, and typical for a hype, there is still too much talk about the technology itself, whereas in the words of Raju and Ram the best AI implementation is “when the customer doesn’t know they are using AI but finds value in the output”. This resonates very well with me, as one of my beliefs is that the customer shouldn’t care about the technology that is used to achieve the desired outcome, within some constraints like legality, ethics, and efficiency, of course.

Zoho is a technology vendor with a focus on business applications. So, Zoho quite quickly realized that consumer type AI that e.g., helps with spell checks, or nowadays research, suffers from two fundamental flaws: lacking privacy/security and accuracy when it comes to business applications. Both violate some of Zoho’s core tenets, namely their pursue of privacy and business applications that offer a lot of value to the customer. Take the example of improving one’s writing – for some years now, this is offered by Zoho Writer, instead of making the user copy and paste potentially sensitive information into an external tool with potentially questionable guardrails.

Similarly for the use of generative AI, or AI in general, in a business context. LLMs, as they are offered by vendors, regularly miss the business context. This means that their responses to a prompt are less than accurate – they are hallucinating. AI works best in context, in this instance in a business context – your business’s context. This requires a process called grounding, or RAG – retrieval augmented generation.

Or in plain words: It requires business data and analytics on it.

In Zohos words, this is contextual intelligence with decision intelligence adding on to it. Decision intelligences creates recommendations and/or actions based upon the findings. Raju Vegesna used a revenue and profit timeline to make this point. Business analytics shows a drop in profit. AI is perfectly able to identify this anomaly. But the real questions are why and what to do. The contextual intelligence correlates this drop to other factors, it gives a diagnosis, and the decision intelligence gives a recommendation on what can (or should?) be done, based on the diagnosis.

Which brings us back to the topic of large language models not being a silver bullet. Language models, especially multi modal ones, can increase the accuracy of e.g., the OCR of an expense application significantly. Ram and his team observed that, in the case of a receipt only having a merchant logo instead of its name, the accuracy of identifying the merchant “has gone up from somewhere {around] 75% to 98%” because of using a multi-modal models. Even a small language model already increases the OCR accuracy, e.g, by not confusing the cash given with the sales price. Scale this up. This increase in accuracy translates to a far higher efficiency of the expense process. And now look at the end-to-end expense process. It starts by taking a picture of the receipt and then extracts and infers different information. On submission the expense report is analyzed for anomalies and policy violations. This multi-step process can be efficiently handled by a variety of specialized models, from narrow models (OCR), small language models (text extraction and inference), mid-sized models (anomaly detection) to LLMs (policy violation. 

Another example is in the legal field, using electronic contract signature as an example. Again, a multi-step process is followed that leads from phishing detection via a narrow model, possibly translation via a small language model, summarization, and anomaly detection via a medium language model and finally the development of recommendations if the suggested document is not compatible with policies.

Similarly, a warranty process in customer service.

All these AI supported processes have two things in common: the user is oblivious to the use of AI and secondly, the orchestrated interaction of smaller models makes the process more resource efficient without endangering process efficiency. 

Ram maintains that “Even though these models are trained using energy intensive CPUs, we are able to run the inference on CPUs, and that that means I'm contributing positively to the enrollment. And again, I'm not passing on the GPU tax to my customer. So, find out models that work for different levels. I mean, it's okay, three to 5 million model is more than enough to identify the name of the merchant on a receipt. And a 50 billion model is enough to rephrase legal statements given its specialized in that domain, that domain and the context is what is helping us to lower the model size and thereby keep costs in check.”

In my opinion, Zoho is right in saying that AI models are going to be commoditized. Leveraging AI in a business application is already now a table stake.

But what about AI and ethics, especially when it comes to training models? After all, Zoho takes a strong position on privacy.

Ram extends this into the realm of training models. “We strongly believe that your data is your data. And it should work just for you, meaning you cannot use your data to train a third party's AI and in turn, you get some subsidized software. And that's not how it should work. Because that's your business secret, right? That's your secret sauce. And you don't want it to be taken over by some model. So, we have set clear policy privacy guidelines, where we have built foundational models that are independent of any of our customer data. And then they are fine tuned to individual customers.” He insists that “if I'm going to use the model that is trained on one company, for their competitor, who has just signed up for CRM, I'm basically selling their data. We don't do that. So, all of these privacy policies intact, and we also have a strong policy towards making our AI bias free.”

On top of this, Zoho makes sure that there is no PII used for learning. Drift is controlled via regular systems audits using split systems. 

Zoho promises to do it right, which is, as Ram says, the promise that every vendor should give and keep. 

Now, that we are in the middle of – or hopefully closer to the end of – a general hype that was caused by Open AI’s ChatGPT, it is time to reemphasize on what is possible and what is not, what should be done and what not. It is time to look at business use cases that are beyond the hype and that can be tied to actual business outcomes and business value.

This, especially, in the light of the probably most expensive demo ever, after Google Bard gave a factually wrong answer in its release demo. A factual error wiped more than $100bn US off Google’s valuation.

I say this without any gloating. Still, this incident shows how high the stakes are when it comes to large language models, LLM. It also shows that businesses need to have a good and hard look at what problems they can meaningfully solve with their help. This includes quick wins as well as strategic solutions.

From a business perspective, there are at least two dimensions to look at when assessing the usefulness of solutions that involve large language models, LLM.

One dimension, of course, is the degree of language fluency the system is capable of. Conversational user interfaces, exposed by chatbots or voice bots and digital assistants, smart speakers, etc. are around for a while now. These systems are able to interpret the written or spoken word, and to respond accordingly. This response is either written/spoken or by initiating the action that was asked for. One of the main limitations of these more traditional conversational AI systems is that they are better in understanding than in – lacking a better word – expressing themselves. Relying on well-trained machine learning models, they are also quite regularly able to surface a correct solution for problems in the problem domain that they are trained for. They usually work based on pretrained intents.

And, based on the training data, they usually give quite accurate responses to questions in their domain.

The problem: They are usually limited to a fairly small number of domains.

LLMs, on the other hand, are generally trained “to understand the relationships between words, phrases and sentences in a language. The goal is to have the LLM generate outputs that are semantically meaningful and reflect the context of the input.” This is part of ChatGPTs answer to the question what the purposes of an LLM is. The training set of an LLM is usually a vast amount of “real world” knowledge that usually comes from publicly available sources – aka the Internet. The output itself can be in written, graphical or other formats.

What LLMs excel in is generating responses to questions in a human way. And they can respond to a wide variety of topics. When focusing on text, they are built to generate coherent and meaningful responses.

The problem: They sometimes lack accuracy and give wrong output with full confidence. Even worse, wrong or inaccurate output is not easily identifiable by a user without the requisite knowledge. Again, refer to the Google Bard example that (temporarily, at least) wiped off $100 billion US from Googles valuation. Not picking on Google, there are plenty of examples around that call out ChatGPT or You.com or other tools.

Consequently, the other dimension to look at is accuracy.

The question is whether both dimensions always matter equally or not. In a business sense, one can argue that accuracy matters always. Receiving factual errors in a business conversation is not only a poor customer experience but may in extreme cases even lead to legal issues.

What is also important to understand is that the more accuracy is required the more the necessity of integrating additional systems to augment the LLM increases. An LLM on its own is not much more than some form of entertainment. Even in search engines, LLMs only augment the search by enabling natural language queries and the delivery of results in human language instead of a mere link list.

At least they should do this.

With all this being said, what are business use cases involving a large language model? As said, there needs to be a reasonable accuracy. Obviously, they require fluency as a precondition, as fluency is the core differentiator of an LLM.

Let’s look at some use cases in no particular order of priority.

• I’d start with something that I’d call “storytelling”. This is basically the creation of market-relevant documents that describe the capabilities and differentiating factors of a product, solution, or service. Being somewhat marketing related (no offence intended) and a first point of contact for customers, it needs to be easy to understand without requiring a great deal of technical accuracy. At the same time, it must not be wrong. A stripped-down version of this could be the (improved) generation of social media content, e.g., tweets. Benefits are faster creation of high-level content for general websites but also, more specifically, for ABM scenarios and landing pages. To be able to create this text, an LLM needs to be connected to internal systems holding requirements, specifications as well as communications between the involved persons. This is also a use case that should be implement-able near-term.
• One of the main tasks of people is the writing of, and more so, responding to emails. Especially, in sales scenarios, customer inquiries can get formulated and suggested based upon previous emails and the context given by the CRM system, e.g., about proposals made. This scenario would already require quite a high accuracy to avoid sending out faulty information that might be legally binding. The benefit of this scenario is a significant reduction time needed to send emails, resulting in increased productivity. It is a scenario that Microsoft has already implemented in its Viva Sales solution.
• Generation of documentation is a scenario that somewhat varies in the requirement for fluency. It can be mainly divided into technical and user documentation. While user documentation needs to be extremely readable, the writing style is somewhat less important for technical documentation. Conversely, technical documentation likely needs to have a high degree of technical accuracy that is not needed in user documentation, which means that either different repositories or different parts of source documents need to be used to create the texts and potentially diagrams and images.
• One of the most promising use cases in the short term is customer service, including enterprise search. Here, users want answers to their questions, not just links or something actioned. To achieve this, it is necessary to connect to a conversational AI, business systems and a well-functioning knowledge base that helps in generating accurate answers when searching for something. The actioning of issues is very similar to what conversational AIs do already now. The differences are that the intent detection can be far better as the LLM can create more than enough training sets for this and that the answers given by the system are far more fluent. The same holds true for an inquiry scenario. However, as a word of caution, the accuracy of responses to inquiries depends heavily on the kb content that gets searched by the enterprise search. Therefore, the kb needs continuous and rigorous scrutiny. If this is given, the benefits lie in increased call deflection and customer satisfaction. Properly implemented, benefits include an improved call deflection as more cases can get handled by the system, combined with an increased customer satisfaction as issue handling can become quite easy and efficient for the customer. Cognigy has recently presented some very good examples (here and here) that also include voice in- and output.
• Agent assistance is somewhat easier to implement, as it mostly needs to connect to the customer service application, including the chat history. Having complete access to sales and marketing data, of course is helpful, too. Combined with a sentiment analysis, the LLM can suggest text blocks for the agent to use. The benefits of this are an increased agent efficiency and quite possibly also higher customer satisfaction as the text blocks do exhibit more empathy with the customer’s situation than texts generated without an LLM.

In summary, these five scenarios show use cases involving an LLM that are beyond the hype. They can get implemented in a short time and they can also be easily tied to business outcomes. That way, their benefits can get measured.

Which other use cases do you see? And how would you tie them to business value?