Monitoring Proxmox VE With Prometheus And Grafana

Подписаться 12 тыс.

Просмотров 12 тыс.

50% 1

Monitoring computers is very important and even more so for hypervisors like Proxmox VE
Because if something goes wrong it will likely affect all of the virtual computers that are being run on that physical computer
Now an interesting open source combination of monitoring tools that's available for free is Prometheus and Grafana
So in this video, we show how to configure Prometheus and Grafana to monitor Proxmox VE, when you're using Docker
=============================
SUPPORT THE CHANNEL
Donate through Paypal:
paypal.me/DavidMcKone
Donate through Buy Me A Coffee:
buymeacoffee.com/dmckone
Become a monthly contributor on Patreon:
/ dmckone
Become a monthly contributor on RU-vid:
/ techtutorialsdavidmckone
==============================
=============================
MY RECORDING HARDWARE:
Blue Yeti USB Microphone
amzn.to/3IfL3qm
Blue Radius III Custom Shockmount for Yeti and Yeti Pro USB Microphones
amzn.to/3G3f89P
RØDE PSA1 Professional Studio Arm
amzn.to/3Z3lPBF
Aokeo Professional Microphone Pop Filter
amzn.to/3VuZl9H
Sony Alpha ZV-E10L Mirrorless Camera
amzn.to/3ITHCoU
Elgato Cam Link 4K Capture Card
amzn.to/43CzQaT
Neewer NP-FW50 Dummy Battery Charger Kit
amzn.to/3qp9Q4s
Elgato Key Light Air - Professional 1400 lumens Desk Light
amzn.to/3G81OB9
Neewer 2 Packs Tabletop LED Video Light Kit
amzn.to/3CcuN5O
Elgato Green Screen
amzn.to/3CoJBOL
=============================
==============================
MEDIA LINKS:
Website - www.techtutorials.tv/
Twitter - / dsmckone1
==============================
For more technical information, including commands used, check out our blog post
www.techtutorials.tv/sections...
Useful links:
github.com/prometheus-pve/pro...
grafana.com/grafana/dashboard...
Chapters
00:00 Intro
00:33 Assumptions
00:50 Create Monitoring Account
05:07 Install PVE Exporter
07:07 Configure PVE Exporter
09:37 Configure Prometheus
12:46 Install Grafana Dashboard
14:59 Summary
monitor proxmox with prometheus,monitor proxmox with grafana,monitor proxmox with prometheus and grafana,monitoring proxmox,monitor pve,proxmox prometheus grafana dashboard,proxmox monitoring prometheus,proxmox monitoring grafana,proxmox monitoring api,proxmox server monitoring

Наука

Опубликовано:

2 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 58

@TechTutorialsDavidMcKone 5 месяцев назад

Please be aware there has been a path change for the pve-exporter When I first set this up, the suggested volume mapping was /path/to/pve.yml:/etc/pve.yml That''s still working for me, although my container hasn't been updated recently After checking the dev's Github webpage, the suggested volume mapping is now /path/to/pve.yml:/etc/prometheus/pve.yml NOTE the extra prometheus folder in the container path

@lawrencerubanka7087 13 дней назад

Thanks again for your great work! I'm up and running...

@TechTutorialsDavidMcKone 13 дней назад

I am finding the combination of Prometheus and Grafana very useful So good to know this video helped

@marcsmith2505 10 месяцев назад

Just what I have been looking for. Can't wait to watch.

@TechTutorialsDavidMcKone 10 месяцев назад

Hope you find the video useful Fairly straightforward to setup

@rmachuca 10 месяцев назад

Thank you, great on sharing the rationale of your choices as each environment is different and it may not apply to each individual use case, so you taking the time to explain allows the end user to take an informed decision. Keep it up!

@TechTutorialsDavidMcKone 10 месяцев назад

Thanks for the feedback, really appreciated

@DarrylGibbs 10 месяцев назад

Great stuff! About to setup Grafana for the first time on my new Proxmox server, and this is going to help!

@TechTutorialsDavidMcKone 10 месяцев назад

I like that it can be run in a container. And having community dashboards makes setting this up a lot easier

@andymok7945 6 месяцев назад

Thanks for making and sharing this video. I did some exporting of metrics in a different method a while back. Will have to lookup what I did and compare to you video to see the differences.

@TechTutorialsDavidMcKone 6 месяцев назад

Thanks for providing feedback, always appreciated

@ethanvolk7292 4 месяца назад

Great video! Made this easy to setup.

@TechTutorialsDavidMcKone 4 месяца назад

I do like this combination for monitoring So good to know the video was useful

@ooocarneiro 8 месяцев назад

Thank you! This was very useful!

@TechTutorialsDavidMcKone 8 месяцев назад

Good to hear the video was helpful

@krishnachinnasamy2764 10 месяцев назад

Great Video. Thank you!

@TechTutorialsDavidMcKone 10 месяцев назад

Thanks for the feedback, always appreciated

@BillyDickson 10 месяцев назад

Thanks David, much appreciate and on my list. This weekend its guacamole docker to nginx proxy manager to cloudflare tunnel with email authentication. 👍

@TechTutorialsDavidMcKone 10 месяцев назад

Sounds like quite the weekend

@fvgoya 10 месяцев назад

That's interesting. I didn't realized that we need to use this node exporter. I don't remember others creators mentioned it. I though in the end will be running only 2 container and not 3 (Exporter, Grafana and Prometheus). As always, thank you David!!!!

@TechTutorialsDavidMcKone 10 месяцев назад

There are other ways but I think this exporter is very useful No agent needed on the nodes so less things to go wrong

@sarmadahsan869 10 месяцев назад

Amazing, Would install in my HomeLab

@TechTutorialsDavidMcKone 10 месяцев назад

Definitely a good open source monitoring solution and it's free to use

@ffftube-le8np 10 месяцев назад

Interesting, I monitor my proxmox using observium via snmp but this looks like a cool alternative to that setup with some advantages. Thank you

@TechTutorialsDavidMcKone 10 месяцев назад

I've tried a lot of monitoring tools over the years but I do like how much easier this is to setup A backup and restore should be a lot easier too as I don't really need the data, I just need the docker compose and config files to get back up and runnning So far, the devs for the exporters I've been using provided good examples to at least get started and the community dashboards for grafana saved a lot of time SNMP can be a pain to setup if you don't already know the mibs but for now that's just going be needed for the physical switches

@scoutguard3015 10 месяцев назад

Thank you!

@TechTutorialsDavidMcKone 10 месяцев назад

Good to know the video was helpful

@lawrencerubanka7087 13 дней назад

Thanks

@TechTutorialsDavidMcKone 13 дней назад

Thank you very much

@cyberjohn44 5 месяцев назад

Nice video. Had some headaches around the new file path but got it working. Very nice dashboard with some very good metrics. One item to note: if you are using a drive point for your VM storage you will need give permissions to it as well. Also I had to use the username and password for the connection in the pve yaml file since I had multi Proxmox servers. Couldn't get the different token keys to work.

@TechTutorialsDavidMcKone 5 месяцев назад

It's unusual for a path change like that It would be good if you could edit a video to make a slight change, like add some text, but the only option is to edit your own copy of the original file and upload it as a replacement I've tried a few monitoring solutions over time though and I do like this Prometheus and Grafana combo

@cyberjohn44 5 месяцев назад

@@TechTutorialsDavidMcKone Agree. It is a good centralized dashboard for monitoring all my Proxmox servers and able to get a quick over view. Love the simple but easy to read metrics to able to look for any potential issues.

@Anuitu2u 5 месяцев назад

@@cyberjohn44 Hi, John. I've been monitoring my Proxmox using influxDB, because, well, it has build in feature (metrix server). And I use dashboard 18621, but it lacking something, so I copy-paste from 15356, so I get its storage pool and modified it to using gauge. But, data from metric server is still missing something. 2 things. 1. temperature sensors. and I use telegraf on LXC to get CPU temperature. 2. Proxmox actual free memory, which is memfree - (cache+buff), which crucial in my opinion. is dashboard 10347 in the video have that (2 things)? is it possible to do that?

@cyberjohn44 5 месяцев назад

@@Anuitu2u Here are the 3 dashboards I am using. Node Exporter Full - 1860 Proxmox via Prometheus - 10347 Docker monitoring with node selection - 8321

@shadowwizzard605 7 месяцев назад

I followed these exact steps, but for some reason my prometheus only shows Prometheus up, it shows neither the nodes nor the exporter

@TechTutorialsDavidMcKone 7 месяцев назад

Check the status of the containers to see if they are up and running, for example docker ps -a Also check the configs, for example, the IP addressing will probably be different for your network And check the logs of the containers to see what they're reporting, for example docker container logs node_exporter

@manuelcappiello8956 2 месяца назад

Thank you! It was informative and very easy to follow. How can we apply this to a second pve server? Do we need to have a cluster or should we create another node-exporter and pve-exporter in the second pve server?

@TechTutorialsDavidMcKone 2 месяца назад

pve-exporter only needs to run on your docker server. It connects to the nodes over the network through an API Add each node you want to monitor to the prometheus job that you setup for prometheus to connect to the pve-exporter Bear in mind, when you have a cluster, pve-exporter will gather the same information from each node So you'll see the same details in Grafana But you still want to monitor all nodes in a cluster for redundancy reasons If you want details at a Linux level, then that's where node-exporter comes in But it needs to be installed on the host you want to monitor So if you want to use it to monitor your PVE nodes then install it on each node and configure prometheus to gather the data from each exporter

@kennynichols5472 6 месяцев назад

Excellent video, I used your tutorials on setting up Docker, Heimdall, Prometheus, and Grafana. I was attempt to add the ProxmoxVE monitor but I'm running into an issue with the docker container restarting after ~1minute. Any thoughts or tips on where to look?

@TechTutorialsDavidMcKone 6 месяцев назад

Good to know the videos have been helpful If a container is restarting then check the container's log file, for example docker container logs prometheus It should tell you what the problem is and which line to check But chances are there's a typo in the script, even if it's just a space missing or an extra one added Or maybe it's pointing to something which doesn't exist e.g. a folder that needs mapping

@JD-ow2rm 5 месяцев назад

Excellent video. I experienced a similar issue. This may help. I ran >docker compose up which enables me to see all the detail of the containers starting. I noticed this error under pve-exporter FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml' I change the volume entry for pve-exporter to : volumes: - ./pve/pve.yml:/etc/prometheus/pve.yml This was successful. Good luck!

@user-hz5hf3qi7x 10 месяцев назад

Hi David, thanks for the interesting video. I tried to follow your instructions, but I have issues to start the pve-exporter. I always get the error message: IsADirectoryError: [Errno 21] Is a directory: '/etc/pve.yml' . In which directory is the file pve.yml stored? Where is my error? Thanks for your support David.

@TechTutorialsDavidMcKone 10 месяцев назад

Sounds like the mapping isn't lining up In the folder where the docker-compose.yml file is and where docker is being run from, I created a sub-folder called pve In that folder I created a file called pve.yml The docker compose file maps /etc/pve.yml to that file I created by referencing ./pve/pve.yml The period or dot is important because it references the current folder and it's a relative path i.e. relative from where docker is running from So the full line for mapping the config file is - ./pve/pve.yml:/etc/pve.yml Do check the syntax, which folder you're in and which folder you're running docker from Otherwise you could try an absolute path such as /home/docker/pve/pve.yml for instance if you are running docker from a home directory for a user called docker

@user-hz5hf3qi7x 10 месяцев назад

Hello David, thank you very much for your help. Now it works.@@TechTutorialsDavidMcKone

@bogdandotct4863 5 месяцев назад

For anyone that hits the issue with node-explorer container not starting due to error "pve-exporter | FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml'" i did the following to fix it: -> edit docker-compose.yml and change pve-exporter volumes path from "/etc/pve.ym" to "/etc/prometheus/pve.ym". It should look as this: volumes: - './pve/pve.yml:/etc/prometheus/pve.yml'

@TechTutorialsDavidMcKone 5 месяцев назад

I take it, you're monitoring PVE with node-exporter? I ask because you're referencing a folder called /etc/pve which I haven't referred to but would exist on a PVE server I didn't use that method in this video, instead I set up monitoring through the API to avoid software installation on PVE Although node-exporter would make sense for additional monitoring

@bogdandotct4863 5 месяцев назад

Hi David!@@TechTutorialsDavidMcKone Yes, i followed your guide and got all the containers up yesterday but i got stuck later, at grafana not importing the dashboard profile. I will try to fix it later today. But, first of all: thank you so much for this guide, it helped me a lot, your brought me quite some time with this great guide. Regarding the error mentioned above, i hit it after configuring the node-explorer container (see instructions around minute 6:45 of your video): the external path is ./pve/pve.yml which your guide said it will map it internally to /etc/pve.yml. But in my case, with your config instructions, the pve-exporter container did not start and was complaining with the error "pve-exporter | FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml' (docker compose logs). So, my workaround was to edit docker-compose.yml change the internal path from "/etc/pve.yml" to "/etc/prometheus/pve.yml"; the pve-exporter contaner ran without other issues after this change.

@TechTutorialsDavidMcKone 5 месяцев назад

@@bogdandotct4863 That seemed odd as I checked my pve-exporter config and it's - ./pve/pve.yml:/etc/pve.yml I've had a look on the Github page and the path is now /path/to/pve.yml:/etc/prometheus/pve.yml So the devs must have changed the internal folder at some point, grrr Thanks for sharing as like others, I'll run into this myself the next time my container is updated

@zyghom 10 месяцев назад

I am very surprised that InfluxDB cannot be used ;-(

@TechTutorialsDavidMcKone 10 месяцев назад

I did look into exporting metrics from Proxmox VE and I know it can send to InfluxDB and Graphite but I'm more familiar with pull models What puts me off push models is it requires punching a hole through the firewall to allow access to the monitoring server which I wouldn't been keen on from a security perspective I think Telegraf would make more sense than Prometheus for InfluxDB, although there is Prometheus exporter for it github.com/prometheus/influxdb_exporter

@zyghom 10 месяцев назад

@@TechTutorialsDavidMcKone thank you. I have used Home Assistant integration that was pulling data from Proxmox directly to InfluxDB and then to Grafana

@fmslick7586 6 месяцев назад

This is a REALLY BAD how to video!! REDO IT

@TechTutorialsDavidMcKone 6 месяцев назад

Can you be more specific please? I need at least some points and suggestions

@fmslick7586 6 месяцев назад

Show or tell file path (/home/username/xyz.abc) .. link to your doc/blog about the video then Chapters OR Chapters than link to blog before asking for money aka support is always better. @@TechTutorialsDavidMcKone

@jpb2085 7 месяцев назад

Very nice tutorial. Despite havin followed along, have been unable to get it working. prometheus exporter is able to scrape data from prox mox (can check the exporter pve target and see scraped metrics), but prometheus does not load the scrape_config for pve, nor do I see any pve endpoint target in prometheus.

@TechTutorialsDavidMcKone 7 месяцев назад

I would suggest checking the config for Prometheus as it sounds like it isn't aware of the exporter It also needs to be reloaded if the config has been changed Otherwise it would be best to check the log file as any errors would show up there

@jpb2085 7 месяцев назад

@@TechTutorialsDavidMcKone Hey Dave, my bad, incorrectly mounted volume for prometheus. Going into running prometheus container and checking prometheus.yml showed me it was using the default, not my custom one from host. All good, thanks again!

@jpb2085 7 месяцев назад

@@TechTutorialsDavidMcKone oh, and subbed!