The main subjects of the conference were:

• Voice modality

• Latency (!!!)

• Reliability

• Cost Cutting

However, the most interesting part was the discussion with other participants. It was good to hear what tools are common in production from so many companies. I wrote down some notes:

In fact, the vast majority of developers use tools from langchain-ai:

• Langchain - here very often the opinion was - "we tried it once, we rejected it, we tried again, now we use it"

• Langgraph (which was a pleasant surprise for me)

(So Eli Brosh - I think there might be some method in the madness in the story you told me :D)

Less frequently, but still relatively common, were solutions such as:

• DsPy (that was a suprise for me)

• Haystack

• OpenRouter

• Langsmith (a lot of people have custom evaluation solutions, I wonder how OpenAI Evals will affect it :) )

Among the big no-showers in production, I noted:

• - LLamaIndex

• - Perplexity API - (which was a huge surprise to me, given the speed and price of the models in the "-online" version)

• - MLFlow

As for on-premise providers, rather unsurprisingly:

• OpenAI (duh... 😅 )

• Claude 3.5-sonnet, mentioned basically in one breath with the subject of code generation

• Google Gemini - chosen mainly in applications benefiting from extreme context size

Only once did I happen to hear someone use on-premise models from Mistral AI. AI21 Labs basically universally had a bad reputation, unfortunately.