4 ms·

New tools and features in the Responses API

74 points by meetpateltech 1y ago

skeptrune 1y ago
Wow background mode looks awesome. I'm excited to work that into our UX for people. Live Q&A is such a dead interface at this point. Reasoning summaries also look great. Anything that provides extra explainability is a win in my book.
pizzuh 1y ago
It's great to see more and more adoption for MCP. I'm not sure it's the most bulletproof protocol, but it feels like it's in a strong lead, especially with OpenAI support. I've been using Codex for the last 24 hours, and background mode boosts your output. You can have Codex work on n+ features async. I had it building a database model alongside frontend authentication, and did both pretty well.
tedtimbrell 1y ago
Im quite surprised they’re actually going with hosted mcp versus just implementing the mcp server locally and interacting with the api
- nknj 1y ago
  you can use local mcp servers with the agents sdk: https://openai.github.io/openai-agents-python/mcp/ https://openai.github.io/openai-agents-python/mcp/ responses api is a hosted thing and so it made most sense for it to directly connect to other hosted services (like remote mcp servers).
jasongill 1y ago
I wish OpenAI would provide more clarity about the Assistants API deprecation, which has been announced as being sunset in spring of 2026 and replaced by the Responses API, but still no other updates on the timeline or migration plan. Prior to the release of the Responses API, the Assistants API was the best way (for our use cases, at least) to interact with OpenAI's API, so hopefully some clarity on the plan for it is released soon (now that Responses API has some of the things that it was previously missing)
- nknj 1y ago
  I hear you and really appreciate the patience here. We're almost ready to share a migration guide. Today, we closed the gap between Assistants and Responses by launching Code Interpreter and support for multiple vector stores in File Search. We still need to add support for Assistants and Threads objects to Responses before we can give devs a simple migration path. Working on this actively and hope to have all of this out in the coming weeks.
  - alasano 1y ago
    Interesting that you're migrating assistants and threads to the responses API, I presumed you were killing them off. I started my MVP product with assistants and migrated to responses pretty easily. I handle a few more things myself but other than that it's not really been difficult.
  - beklein 1y ago
    On the announcement page they are saying that "...introducing updates to the file search tool that allow developers to perform searches across multiple vector stores...". On the docs, I still find this limitation: "At the moment, you can search in only one vector store at a time, so you can include only one vector store ID when calling the file search tool." Anybody knows how searching multiple vector stores is implemented? The obvious plan would be to allow something like: "vector_store_ids": ["<vector_store_id1>", "<vector_store_id2>", ...]
    - nknj 1y ago
      sorry about the error in the docs. we're removing that call out. `"vector_store_ids": ["<vector_store_id1>", "<vector_store_id2>"]` is exactly right. only 2 vector stores are supported at the moment.
      - akgfab 1y ago
        2 feels quite arbitrary and honestly not that much of an improvement. Any plans to up that limit?
andrewrn 1y ago
It was never really clear what the difference between the chat and responses APIs were. Anyone know the difference?
- brittlewis12 1y ago
  chat completions is stateless — you must provide the entire conversation history with each new message; openai stores nothing (at least nothing that the downstream product _can use_) beyond the life of the request. responses api, by contrast, is stateful — only send the latest message, and openai stores the conversation history, while keeping track of other details on behalf of the calling app, like parallel tool call states. but i would say that since chat completions has become an informal industry standard, the responses api feels like an attempt by openai to break away from that shared interface, because it is so easy to swap out providers with nothing more than a base url and a model id, to a paradigm which requires data migration as well as replacement infrastructure (containers for code execution, for example).
  - nknj 1y ago
    one additional difference between chat and responses is the number model turns a single api call can make. chat completions is a single turn api primitive -- which means it can talk to the model just once. responses is capable of making multiple model turns and tool calls in a single api call. for example, you can give the responses api access to 3 tools: a vector store with some user memories (file_search), the shopify mcp server, and code_interpreter. you can then ask it to look up some user memories, find relevant items in the shopify mcp store, and then download them into a csv file. all of this can be done in a single api call that involves multiple model turns and tool calls. p.s. - you can also use responses statelessly by setting store=false.
    - OutOfHere 1y ago
      What are my choices for using a custom tool? Does it come down to: function calling (single turn), MCP (multi-turn via Responses)? What else are my choices? Why would anyone want to use Responses statelessly? Just trying to understand.
rafram 1y ago
> Encrypted reasoning items: Customers eligible for Zero Data Retention (ZDR) can now reuse reasoning items across API requests So, so weird that they still don't want you to see their models' reasoning process, to the point that even highly trusted organizations with ZDR contracts only get them in a black-box encrypted form. Gemini has no issue showing its work. Why can't OpenAI?
- vessenes 1y ago
  Is this true? I can click open o3’s dialogue and see a running monologue. I guess it might be a summary of the actual reasoning though.
  - hhh 1y ago
    It is a summary
  - mediaman 1y ago
    Correct, you are not seeing the reasoning chains.
    - rafram 1y ago
      I may be giving Gemini too much credit, actually - seems like its "reasoning" may be a summary as well.
      - Doohickey-d 1y ago
        They changed it yesterday or so: it used to show the actual reasoning, now it no longer does. And the reasoning was quite useful to see if it was going down the wrong track, the summary is much less so.
        epiccoleman 1y ago
        That's disappointing. I was getting a lot of utility from reading through the thoughts returned by Gemini when I used it in Cursor - occasionally even learning something new from its stream of "consciousness". Obfuscating the information because it can be used to train competitors seems misguided, if understandable.
orasis 1y ago
Reasoning models can now call tools during the reasoning process.
joshwarwick15 1y ago
List of remote MCP servers to use here: https://github.com/jaw9c/awesome-remote-mcp-servers https://github.com/jaw9c/awesome-remote-mcp-servers

74 points by meetpateltech 1y ago

Wow background mode looks awesome. I'm excited to work that into our UX for people. Live Q&A is such a dead interface at this point. Reasoning summaries also look great. Anything that provides extra explainability is a win in my book.

pizzuh 1y ago

It's great to see more and more adoption for MCP. I'm not sure it's the most bulletproof protocol, but it feels like it's in a strong lead, especially with OpenAI support. I've been using Codex for the last 24 hours, and background mode boosts your output. You can have Codex work on n+ features async. I had it building a database model alongside frontend authentication, and did both pretty well.

tedtimbrell 1y ago

Im quite surprised they’re actually going with hosted mcp versus just implementing the mcp server locally and interacting with the api

nknj 1y ago
you can use local mcp servers with the agents sdk: https://openai.github.io/openai-agents-python/mcp/ https://openai.github.io/openai-agents-python/mcp/ responses api is a hosted thing and so it made most sense for it to directly connect to other hosted services (like remote mcp servers).

jasongill 1y ago

I wish OpenAI would provide more clarity about the Assistants API deprecation, which has been announced as being sunset in spring of 2026 and replaced by the Responses API, but still no other updates on the timeline or migration plan. Prior to the release of the Responses API, the Assistants API was the best way (for our use cases, at least) to interact with OpenAI's API, so hopefully some clarity on the plan for it is released soon (now that Responses API has some of the things that it was previously missing)

nknj 1y ago
I hear you and really appreciate the patience here. We're almost ready to share a migration guide. Today, we closed the gap between Assistants and Responses by launching Code Interpreter and support for multiple vector stores in File Search. We still need to add support for Assistants and Threads objects to Responses before we can give devs a simple migration path. Working on this actively and hope to have all of this out in the coming weeks.
- alasano 1y ago
  Interesting that you're migrating assistants and threads to the responses API, I presumed you were killing them off. I started my MVP product with assistants and migrated to responses pretty easily. I handle a few more things myself but other than that it's not really been difficult.
- beklein 1y ago
  On the announcement page they are saying that "...introducing updates to the file search tool that allow developers to perform searches across multiple vector stores...". On the docs, I still find this limitation: "At the moment, you can search in only one vector store at a time, so you can include only one vector store ID when calling the file search tool." Anybody knows how searching multiple vector stores is implemented? The obvious plan would be to allow something like: "vector_store_ids": ["<vector_store_id1>", "<vector_store_id2>", ...]
  - nknj 1y ago
    sorry about the error in the docs. we're removing that call out. `"vector_store_ids": ["<vector_store_id1>", "<vector_store_id2>"]` is exactly right. only 2 vector stores are supported at the moment.
    - akgfab 1y ago
      2 feels quite arbitrary and honestly not that much of an improvement. Any plans to up that limit?

andrewrn 1y ago

It was never really clear what the difference between the chat and responses APIs were. Anyone know the difference?

brittlewis12 1y ago
chat completions is stateless — you must provide the entire conversation history with each new message; openai stores nothing (at least nothing that the downstream product _can use_) beyond the life of the request. responses api, by contrast, is stateful — only send the latest message, and openai stores the conversation history, while keeping track of other details on behalf of the calling app, like parallel tool call states. but i would say that since chat completions has become an informal industry standard, the responses api feels like an attempt by openai to break away from that shared interface, because it is so easy to swap out providers with nothing more than a base url and a model id, to a paradigm which requires data migration as well as replacement infrastructure (containers for code execution, for example).
- nknj 1y ago
  one additional difference between chat and responses is the number model turns a single api call can make. chat completions is a single turn api primitive -- which means it can talk to the model just once. responses is capable of making multiple model turns and tool calls in a single api call. for example, you can give the responses api access to 3 tools: a vector store with some user memories (file_search), the shopify mcp server, and code_interpreter. you can then ask it to look up some user memories, find relevant items in the shopify mcp store, and then download them into a csv file. all of this can be done in a single api call that involves multiple model turns and tool calls. p.s. - you can also use responses statelessly by setting store=false.
  - OutOfHere 1y ago
    What are my choices for using a custom tool? Does it come down to: function calling (single turn), MCP (multi-turn via Responses)? What else are my choices? Why would anyone want to use Responses statelessly? Just trying to understand.

rafram 1y ago

> Encrypted reasoning items: Customers eligible for Zero Data Retention (ZDR) can now reuse reasoning items across API requests So, so weird that they still don't want you to see their models' reasoning process, to the point that even highly trusted organizations with ZDR contracts only get them in a black-box encrypted form. Gemini has no issue showing its work. Why can't OpenAI?

vessenes 1y ago
Is this true? I can click open o3’s dialogue and see a running monologue. I guess it might be a summary of the actual reasoning though.
- hhh 1y ago
  It is a summary
- mediaman 1y ago
  Correct, you are not seeing the reasoning chains.
  - rafram 1y ago
    I may be giving Gemini too much credit, actually - seems like its "reasoning" may be a summary as well.
    - Doohickey-d 1y ago
      They changed it yesterday or so: it used to show the actual reasoning, now it no longer does. And the reasoning was quite useful to see if it was going down the wrong track, the summary is much less so.
      - epiccoleman 1y ago
        That's disappointing. I was getting a lot of utility from reading through the thoughts returned by Gemini when I used it in Cursor - occasionally even learning something new from its stream of "consciousness". Obfuscating the information because it can be used to train competitors seems misguided, if understandable.

orasis 1y ago

Reasoning models can now call tools during the reasoning process.

joshwarwick15 1y ago

List of remote MCP servers to use here: https://github.com/jaw9c/awesome-remote-mcp-servers https://github.com/jaw9c/awesome-remote-mcp-servers