Indian AI Infra: Powering the Big Data Applications
Trends, opportunities and challenges for AI Infrastructure startups based out of India
The AI gold rush is upon us. In past couple of months, there has been a surge of interest in AI applications because of the potential shown by LLMs like GPT-3 and Dall-E. However, implementing the industrial applications of AI are quite difficult for businesses and organizations who don’t have expertise/talent/budget to build these applications from scratch but they do have the domain knowledge to harness this tech.
While there is a euphoria about AI applications, a lot of background tasks have to be undertaken by companies before becoming data-ready. So, building a startup which helps companies to become data-ready during this gold rush of AI makes a lot of sense as everyone needs tools/infra for building data/AI applications and this is where the space of Modern Data Stack(MDS) becomes exciting.
Modern Data Stack comprises of collection of tools that are centered around a cloud data warehouse and covers different stages of the data journey from ingestion, storage, transformation to business intelligence.
Some of the most successful AI companies like Snowflake, ScaleAI, Databricks, Confluent, Fivetran, MonteCarlo, and others have taken this path. Basically, it’s better to be a pick and shovel company than a miner during gold rush1.
Because of this, MDS startups have seen explosive growth over the past few years. Over $12 billion has been invested in this space globally in 2020-2021. Unfortunately, coming to India, our share has been <1% in this space in terms of investments.
But still, there are exciting startups coming up in this space like Atlan, Hevo, and AccelData which are building great products for the world from India. Further, based on interactions with various people who are building exciting stuff in this space, I am very sure that there would be many more startups making noises for all the right reasons in next 5 years.
Attached is the landscape in the AI Infrastructure space, where I have tried to map some startups which are building in this space from India for the world.
Some of the trends, opportunities and challenges in India which makes this space exciting are:
Trends and Opportunities:
Fundamental shift to cheaper and simpler data storage technologies (This was not the case earlier with Hadoop)
Increase in BI budgets across organizations
Monolithic ETL → efficient ELT
Increase in demand for real-time streaming technologies for applications such as churn prediction, forecasting, in-app personalisation23
Data warehouses have unlocked an entire ecosystem that revolve around them: ELT, reverse ETL, data quality, metrics stores, augmented analytics, etc.4
Disconnection between data engineers and data analyst is leading to concept of data mesh (Basically, each person handling their own data)
Immaturity in data governance space which becomes critical as data explodes
Taking cues from the internal tools built at FAANGs and making it mainstream (Eg: Kafka was originally developed at LinkedIn)
Increased focus towards Ethical AI
Challenges:
Adoption of AI in India is limited. Major market for this stack is in US and hence, most of the startups are based out there.
Startups need to carve out revenues from limited enterprise budgets. So, the value-add should be substantial (10x improvement over others) and not incremental
VCs aggressively invested in emerging sectors in MDS, due to which some categories went from nascent to crowded rapidly.
Not all organizations would require or have know-how to use sophisticated AI and would be happy with simple predictions from Excel
Overall, it is an exciting space with it’s own fair share of challenges and it would be interesting to see how it evolves in future
Feel free to DM me on LinkedIn if you have any thoughts on the piece or are working on anything in the space
Sources: