Unraid
AI Chat Unleashed: Quick Serge Chat Setup on Unraid!
AI Chat Unleashed: Quick Serge Chat Setup on Unraid!
#Chat #Unleashed #Quick #Serge #Chat #Setup #Unraid
“IBRACORP”
Install Serge Chat, use LLaMA LLM on Unraid easily using Docker, and dive into the world of AI Chat!
This tutorial simplifies the setup process, guiding you through each step to get your chat application up and running.
Perfect for tech enthusiasts and beginners alike, we’ll explore the…
source
To see the full content, share this page by clicking one of the buttons below |
Have you ever imagined chatting with an Alpaca through AI? How do you think it will change your daily communications? Share your thoughts below! 🚀🦙
Nice, i love this channel.. i dont know what i want until you show me 😀
Nice one. I've just started looking into hosting an LLM, I went down the route of a virtual machine running linux on unraid to then do work with AutoGen but it's been a bit of a fuss (I do it all remotely on a tiny laptop screen).
I've just installed this Serge and it seems to be going well. I would prefer it if it had a username and password login screen before just going straight into incase you wanted to open the port to view it remotely.
Do you know if there's a way we can upload PDF documents to our Serge docker for it to read and answer questions on? I've been looking at sooooo many solutions lately and they all seem a bit flawed one way or another.
How about giving it access to documents in order to ask it questions on local data instead of sifting through it manually?
Thanks for covering #Serge. I’ll be spinning this up!
9:23 vote Robert Kennedy Jr 2024
#kennedy2024
#kennedy24
#RFKjr2024
#rfkjr24
The only tyrant we can trust
I tested this and It was kinda frustrating how slow it is.
Question for anyone here that wants to answer? What are your thoughts on using consumer hardware vs EOL enterprise hardware for your unRAID server? My unRAID box is a Super Micro 2U Chassis 12x 3.5" SATA/SAS backplane with a SM mobo and Dual X5680s and 96GB of DDR3 with 10 x 4TB (array), 1 x 4 TB (parity) + 1 x 256gb SSD (Cache / Appdata, Domain array for VM files). I need to upgrade for stability reasons, but mostly due to power consumtion. Should I sell the Super Micro server, for whatever i can get after I update to a new server? I was thinking of just getting a 12900k and 4 x 16tb drives or 4 x 14tb drives and pulling my LSI HBA from the unraid box or from another server I have (an R820, not running due to power bills). Thoughts anyone?
Dude youre back, hows life going?
Thanks for the Tutorial! Any idea how to make it use the GPU ? My GPU is properly setup on my machine, and it doesn't seems to use it, even If I set some GPU layer.
Why is it so slow? Is this a hardware issue? Or an LLM issue?
I wish there were server CPU benchmarks. Like on Unraid how many docker containers, how many VM's how many Plex streams ect.
Thank you for the video, it would be nice to include a link to the repo etc. for Serge
Been using Serge for a few months. You can load models directly from HuggingFace to appdata>serge>weights if you want to try something different. Those models should be GGUF, not split, and append .bin to the file.
You'll also find that some chats respond more quickly than others for a given core count, or bump up your core usage for a faster response. Their recent changes make it difficult to copy/paste from one chat to another. Previously I'd come up with inputs and run multiple chats, copying my input to each so I could monitor how they responded.
This is a great video. I'm happy to see all the LLM available… I heard good things about Mistral and Mixtral, I'll finally will be able to give it a try! Thanks! 🙂
I’d love more content on AI. Image generations in particular would be interesting. Perhaps even comparisons.
Would you recommend Serge over Ollama?