Did you find that some data model patterns were easier to detect for some LLM ? I am curious on how training might have made some agents better at graph navigation for instance?
Yes, we've definitely found that some API data models are easier for models to navigate than others.
The largest factors of Agent inefficiency we've identified so far are:
1. Many APIs lack robust-enough search, forcing agents to page through hundreds or thousands of paginated responses until they find the record they are looking for (our Context Store addresses this).
2. Many APIs have HUGE response sets. Our MCP helps handle this by letting the agent decide exactly what fields they can return.
3. With our SDK, you can literally build your own MCP on top of any source we support (50+ right now and will grow). This is super powerful, and allows you to build more ergonomic MCP servers and tools - even if the models themselves are not intuitive or easy for the LLM to leverage directly.
Combining all three of these together, we see the vast majority of challenges can be addressed via a strong system prompt for guidance. Fine tuning could get you further but anyway, you'd still want your fine tuned model to build on this same foundation, since the efficiences will transfer across use cases and models.
@ecares - Does this answer your question? What do you think?
Your point about search being a bottleneck is spot on. IMO, search APIs should return guidance to agents to help them winnow down the results faster. For example, if your query returns 1000 results, then it should tell the agent, "too many results, we recommend you filter on column X because of Y to improve your search. Here are the possible values in column X: ..."
There are a lot of APIs like this that I really wish would expose downloading a parquet file instead of trying to implement server-side filtering and reporting query features.
Working with APIs is often frustrating and the worst ones are terribly ineficient and frustrating. Our Agent SDK and Agent Context Store insulates you and your agent from this headache, allowing you to query from those synced datasets directly.
The feedback about wanting to download a parquet file is super interesting...
Glad to hear this resonates with you also. We're aiming to give agents more control over their context, and easier access paths regardless of the source system.
As far as I know, modern Node compat in Deno is also quite great - I just import packages via 'npm:package' and they work, even install scripts work. Although I remember that in the past Deno's Node compat was worse, yes.
> Deno is written in Rust, which allows them to move faster and more safely. Contributing to and extending Deno is a breeze. You can add Rust crates to the runtime and use them from JS.
