The Large language models (LLMs) are increasingly being used to create chatbots. As generative Artificial Intelligence (AI) continues to revolutionize the industry, chatbots have gone from representing approximately 18% of the total available LLM applications to now accounting for 46% in less than a year, and this number is only increasing. This was unveiled in the latest “Data Trends 2024” study conducted by Snowflake, the Data Cloud company, which shows that 65% of developers are working on LLM projects for work purposes, indicating a shift in the importance of leveraging generative AI to improve productivity, efficiency, and knowledge for workers.
The results of the “Data Trends 2024” study are based on usage data from over 9,000 Snowflake customers. The report focuses on how global business and technology leaders are leveraging resources like AI to build their data foundation and transform future business operations. The new data shows a shift from LLM applications that required writing (2023: 82%, 2024: 54%) to chatbots that are interacted with through iterative text, providing the ability to have a natural conversation.
“Conversational applications are on the rise because it is the natural way humans interact. And now it is even easier to interact by conversing with an application,” explains Jennifer Belissent, Principal Data Strategist at Snowflake. “We expect this trend to continue as it becomes easier to create and deploy conversational LLM applications, especially knowing that the underlying data remains well-governed and protected. With that peace of mind, these new interactive and highly versatile chatbots will meet both business needs and user expectations.”
More than 33,000 LLM applications in nine months
The study also shows that 20,076 developers from the Snowflake Streamlit developer community have created over 33,143 LLM applications in the past nine months. When it comes to developing AI projects, Python is the preferred programming language for its ease of use, active developer community, and extensive ecosystem of libraries and frameworks. In Snowpark, which allows developers to create applications rapidly and cost-effectively, Python’s usage grew significantly faster than Java and Scala (over the past year): Python grew by 571%, Scala by 387%, and Java by 131%. With Python, developers can work faster, speeding up prototyping, experimentation, and overall learning as development teams venture into cutting-edge AI projects.
In terms of where application development takes place, the trend is towards programming LLM applications directly on the platform where data is also managed. This is indicated by the 311% increase in Snowflake Native Apps- which enables developing applications directly on the Snowflake platform- between July 2023 and January 2024. Developing applications on a single data platform eliminates the need to export data copies to third-party technologies, helping to develop and deploy applications more quickly while reducing operational maintenance costs.
Importance of data governance is growing in companies
With the adoption of AI, companies are increasing their analysis and processing of unstructured data. This allows companies to discover untapped data sources, making a modern approach to data governance more crucial than ever to protect sensitive and private data. The report reveals that companies have increased unstructured data processing by 123% in the last year. IDC estimates that up to 90% of the world’s data consists of videos, images, and unstructured documents. Clean data gives an advantage to linguistic models, so unlocking this untapped 90% opens up a range of business advantages.
“Data governance is not about blocking data, but about unleashing its value,” says Belissent. “We break down governance into three pillars: knowing the data, protecting it, and using it to generate value. Our customers use new features to label and classify data so that appropriate access and usage policies can be applied. The use of all data governance features has increased by 70% to 100%. As a result, the number of queries on protected objects has increased by 142%. When data is protected, it can be used safely. That provides peace of mind.”
“Taken individually, each of these trends is a single data point showing how organizations worldwide are tackling different challenges. When considered together, they tell a broader story about how CIOs, CTOs, and CDOs are modernizing their organizations, addressing AI experiments, and solving data issues, all necessary steps to leverage the opportunities presented by advanced AI,” says Belissent. “The key is to understand that the era of generative AI does not require a fundamental change in data strategy. However, it does require accelerated execution of that strategy. It requires breaking down data silos even faster and opening access to data sources wherever they are in the enterprise or in a broader data ecosystem.”
The complete “Data Trends 2024” study can be downloaded from the following link.