Secrets to Self-Hosting Ollama on a Remote Server
Secrets to Self-Hosting Ollama on a Remote Server
#Secrets #SelfHosting #Ollama #Remote #Server
“Mervin Praison”
👋 Hey Tech Enthusiasts! Today, I’m thrilled to share a complete guide on self-hosting the Llama 3 language model using Google Cloud! Whether you’re using GCP, AWS, or Azure, the concepts remain the same. 🌐💻 Secrets to Self-Hosting Ollama on a Remote Server
🔧 What You’ll Learn:
Creating a Linux VM: How to set up a virtual machine with GPU support on Google Cloud.
Installing Llama: Step-by-step instructions on installing and activating Llama 3 on your VM.
Remote Access Activation: Tips on how to make your Llama server accessible and secure.
UI Integration: How to build and integrate a chatbot user interface to interact with your Llama model.
🎬 In this video, I take you through each step, from VM creation to the exciting part of chatting with your own AI. Don’t miss out on learning how to fully control your AI’s environment and keep your data in-house. Perfect for developers and tech enthusiasts looking for hands-on AI deployment experience!
👍 Like this video if you find it helpful and subscribe to stay updated with more content on artificial intelligence and technology. Ring that bell for notifications on new uploads!
🔗 Resources:
Sponsor a Video:
Do a Demo of Your Product:…
source
To see the full content, share this page by clicking one of the buttons below |
Really enjoying your videos, I struggle to keep up but I am figuring it out. Managed to get llama3 working through the cloud and using open-webui locally too. I'm impressed seeing as though I failed to get chainlit to work. Thanks for the videos, and keep it up. I hope it isn't gonna cost me a fortune now…lol
how much per month for host it? commercial deficit will kill your wallet. no tks.
is the "$204" exactly what you have to pay regardless of how many people use the app per day or per month?
is the API key really just fake api key? we dont need to find where to get the api key?
I love these type of projects!
Remember this setup is not secure until you use whitelisting and auth
Why not use a docker container?
I did same experiment yesterday with aws EC2 instance 32 GB Memory (without GPU), And output was amazing.
Is it possible to make it sass ready?
Make video on hosting ollama for Free Remotely ?
I run Ollama + openen-webui for chat imho provide a more similar experience as chatgpt
Still respect for delivering such content. What I am after is to create an automatic ollama deployment on cloud provider which bills per minute and after the prompt is consumed the machine is shut off so to save on consumption and this way you would be billed for the minutes used – would like to see that
Thanks really amazing! As always. You do a great job.
In 10 mins, you explained what'd take an expert hours. Covered Google Computing Engine, Ollama, app ideas. Your strength is clear content creation. Good work! Kudos! Consider Docker container packaging for one-click installation via gcloud CLI. I'm thinking of working on this and will share it with you and the community once ready.
Nice idea, but surely, it's cheaper, more efficient to use Groq?
Mervin keep up the good work. Significant videos 🎉🎉🎉
I really wanted this video to be traching how to host your own Ollama instance for non tech savy people – I really tried to like this video for what i could be. But sadly the video ist like you deliverd it to us ….. Honest feedback: This video feels more like a G-Cloud ad video with no explanation what so ever …. You leave a lot of unanswered questions: WHY Google-Cloud when the Title is Self-Hosted? (Self-Hosted means YOU host something YOURSELF) Why show 204$ a month first then add options and then place your camera over the final price? Why Ubuntu 22.04 LTS when 23.10 will have never gpu drivers? Why say Copy and Pste 5 times in 15 Seconds … why not just explain what you do or dont say anything? Why not give a "security warning" that copy pasting that ollama command as root could be dangerous, read the script code first? Why choose a Tesla T4? How many Tokens does it have? WHY did you choose this gpu which makes it so expensive? Why did you change for 10GB to 100GB? How many models do you want to host there? How big are the models? Where is the information for the user to make an informed and education decision from you? If you use Chainlit, explain beforehand what this is please. – Nice Security warning around the firewall and network topic, thanks for that! – I really dont want to come over as an a**hole but other videos of you have been much more detail oriented and better planned and excuted for me as a viewer and an DevOps person. Thanks for the time!
Great video! Would love a tutorial on installing a server on your home PC to access from anywhere! Thanks again!
What version of llama are you running 8 or 70? Thank you as well for the great video.
This is great! Have you been able to setup the Ollama Web UI remotely as well?
so much for self-hosting! GCloud!
amazing video ! but you need to turn off your vm !!! :)🤣
Think better to buy or make a machine to host the ai server. But didn’t know there is costlier way to do this via Google cloud . Also was familiar with using openui, but never seen chainlit option. So good value with this vid.. kudos