adilhafeez

6 ms·

1. 1.▲
   Where are people finding GPU capacity?

   5 points by adilhafeez 1mo ago|3 comments
2. 2.▲
   Show HN: Plano – Edge and service proxy with orchestration for AI agents(github.com)

   8 points by adilhafeez 9mo ago|2 comments
3. 3.▲
   Show HN: Claude Code 2.0 router – preference-aligned routing to multiple LLMs(github.com)

   4 points by adilhafeez 1y ago|1 comments
4. 4.▲
   by adilhafeez 1y ago

   Short answer is latency impact is very minimal. We use envoy as request handler which forwards request to local service written in rust. Envoy is proven to be high performance, low latency and highly efficient on request handling. If I have
5. 5.▲
   Show HN: Arch-Router – 1.5B model for LLM routing by preferences, not benchmarks

   66 points by adilhafeez 1y ago|15 comments
6. 6.▲
   You need an out-of-process triage agent(archgw.com)

   3 points by adilhafeez 1y ago|0 comments
7. 7.▲
   by adilhafeez 2y ago

   thanks for sharing - this is quite insightful
8. 8.▲
   by adilhafeez 2y ago

   Hi - this is Adil the co-founder who developed archgw. We are working tirelessly to create a framework that would help developers write agentic application without having to write all the crufty/boilerplate code. At the very minimum we
9. 9.▲
   by adilhafeez 2y ago

   Envoy has proven itself in the industry and we didn't want to reinvent the wheel by doing what envoy had already done for observability, rate-limits, connection management etc. And reason for using proxy-wasm was so we don't take
10. 10.▲
    by adilhafeez 2y ago

    Thanks :) thanks for those comments. Those are great questions. Let me respond to them one by one, > Can I use just the model itself? yes - our models are on huggingface. You can use them directly. > Do you have models hosted somewher
11. 11.▲
    by adilhafeez 2y ago

    Thanks! My responses inline, > do you have to use envoyproxy to use archgw Yes, 100%. Our gateway is implemented as rust filter which runs inside envoy process. > Can archgw be used for LLM routing without using envoyproxy? Unfortunat
12. 12.▲
    by adilhafeez 2y ago

    Thanks! Those are all good questions. Let me respond to them one by one, > Can I just use arch for routing between LLMs Yes, you can use arch_config.yaml file to select between LLMs. In fact we have a demo on llm_routing [1] that you can
13. 13.▲
    Show HN: archgw: open-source, intelligent proxy for AI agents, built on Envoy(github.com)

    20 points by adilhafeez 2y ago|14 comments
14. 14.▲
    by adilhafeez 2y ago

    You are right, since arch is an ingress wasm filter it can be setup inside Istio just like any other envoy filter. You would need to pass arch_config someone which should be easy. We will have samples/demos for Istio and K8s deployment
15. 15.▲
    Show HN: Arch – an intelligent prompt gateway built on Envoy(github.com)

    21 points by adilhafeez 2y ago|19 comments
16. 16.▲
    by adilhafeez 2y ago

    Jailbreak ensures a smooth developer experience by controlling what traffic from user make its way to the model. With jailbreak (and other guardrails soon to be added) developers can short-circuit response and with observability developers
17. 17.▲
    by adilhafeez 2y ago

    You can also see current list of issues at https://github.com/katanemo/arch/issues , and can also post new feature requests and bug fixes there.
18. 18.▲
    Show HN: Arch – an intelligent prompt gateway built on Envoy(github.com)

    31 points by adilhafeez 2y ago|1 comments
19. 19.▲
    by adilhafeez 11y ago

    Good initiative
20. 20.▲
    by adilhafeez 11y ago

    pretty cool