Hi all,

Decided I want to give Hermes a try, but man their documentation is horrid. So, for running fully offline, here’s their “guide”: https://hermes-agent.ai/how-to/run-hermes-offline

Now since that’s trash, anyone here actually doing this? Did you just remove all references to non-local providers in the config.yaml? Are you using other containers for TTS and STT? Any help would be appreciated as I dive in.

Thanks in advance.

    • Fmstrat@lemmy.worldOPEnglish
      1·
      8 hours ago

      Ahhh I had thought they were related because those docs have zero info on fully offline capabilities. :(

      • pech@lemmy.worldEnglish
        1·
        6 hours ago

        The info might be a little scattered throughout the various topics, but they have a suggestion on their “setting the main model” page:

        providers:
  local-vllm:
    api: http://127.0.0.1:8000/v1
    models:
      my-reasoner:
        answer_in_reasoning: true

        This goes in the config.yaml, it used to be ‘base_url’ instead of API, but the structure should be close enough reason through.

        • Fmstrat@lemmy.worldOPEnglish
          1·
          2 hours ago

          Yea, connecting to my local llama instance was pretty straight forward. But then I realized there are still moe providers set up with openai and anthropic outside of the main providers, and then there are MCP tools and skills that don’t ask permission to use the internet, etc. Just seems like OpenWebUI gives you a lot more visibility and control.