Hi Tom,
Unfortunately it’s not working for me.
The graylog container doesn’t start.
Clean ubuntu 22.04 install from VM template on proxmox.
apt installed docker-compose, not snap.
Opensearch container shows errors.
That last log file, /usr/share/opensearch/logs/docker-cluster.log inside the container doesnt’ exist.
OpenSearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/opensearch/data/nodes];
Likely root cause: java.nio.file.AccessDeniedException: /usr/share/opensearch/data/nodes
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397)
at java.base/java.nio.file.Files.createDirectory(Files.java:700)
at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
at java.base/java.nio.file.Files.createDirectories(Files.java:793)
at org.opensearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:322)
at org.opensearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:259)
at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:320)
at org.opensearch.node.Node.<init>(Node.java:478)
at org.opensearch.node.Node.<init>(Node.java:375)
at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
at org.opensearch.cli.Command.main(Command.java:101)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
For complete error details, refer to the log at /usr/share/opensearch/logs/docker-cluster.log
Which causes graylog to not connect and shows this in the logs
2023-05-10 23:44:54,022 ERROR: org.graylog2.storage.versionprobe.VersionProbe - Unable to retrieve version from Elasticsearch node: Failed to connect to opensearch/172.18.0.3:9200. - Connection refused.
Makes leads me to guess that you did not add your user to the docker group and therefore it does not have proper permission to bind service or you have another service running on that some port.
Thanks, I must have done some fat fingers at some point (it was late last night when I did this). I blew away the VM, cloned another one and then followed your instructions, and all is working.
Thank you
EDIT: Tip for any proxmox users that might come here after they get an error, you have to set the CPU as host, as the AVX instruction set is required for one of the containers (can’t remember which right now) and using CPU as KVM won’t work.
With the talk of running containers as root lately, I’m surprised you didn’t use Podman.
I do wish that xcp-ng had container support built in (in 8.3?), that would make life easier for me right now. Otherwise I’m going to do things the hard way and learn my way around Podman (and hope that doesn’t get “Oracled” like CentOS).
“Oracled” refers back to the Open Solaris issue, I think it is a word we need to put into popular tech grammar to remind them how we feel about the issue.
Just a notice to all that run an ancient Intel based motherboard with Proxmox…
The system I used is very reliable but is: “Intel i5-7600 (4) @ 4.100GHz”.
But I get “WARNING: MongoDB 5.0+ requires a CPU with AVX support.”
So I ran thru the tut with success on a new instance on VULTR. Success!
But I likely oversized it at 8MB with 4 CPU cores. I will likely do a system recovery to shrink the disk and create a raw image so that I can stand it up on a much smaller instance likely on RackNerd.
Don’t need horsepower but do want to suck in my pfsense firewall DROP/REJECT logs.
What about this opensearch security plugin? Looks like it’s starting this process and it clearly warns not to open it up to the web or use it as a production machine.
opensearch | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
opensearch | **************************************************************************
opensearch | ** This tool will be deprecated in the next major release of OpenSearch **
opensearch | ** https://github.com/opensearch-project/security/issues/1755 **
opensearch | **************************************************************************
opensearch | OpenSearch Security Demo Installer
opensearch | ** Warning: Do not use on production or public reachable systems **
opensearch | Basedir: /usr/share/opensearch
opensearch | OpenSearch install type: rpm/deb on NAME="Amazon Linux"
opensearch | OpenSearch config dir: /usr/share/opensearch/config
opensearch | OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
opensearch | OpenSearch bin dir: /usr/share/opensearch/bin
opensearch | OpenSearch plugins dir: /usr/share/opensearch/plugins
opensearch | OpenSearch lib dir: /usr/share/opensearch/lib
opensearch | Detected OpenSearch Version: x-content-2.7.0
opensearch | Detected OpenSearch Security Version: 2.7.0.0
opensearch | /usr/share/opensearch/config/opensearch.yml seems to be already configured for Security. Quit.
opensearch | Enabling OpenSearch Security Plugin
opensearch | Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
One Issue I have with Graylog is that it uses the SSPL which is not really open source since it restricts how it can be hosted for customers. if you host it as a service you have to open source any management code you use to setup graylog. It isn’t really clear or tested in court what management source code means or what counts as a service. The possible doomsday scenario is if LTS hosts/manages graylog for a comanaged customer then LTS would have to give the customer access to source code for the tools that made that service available to the customer which might not be possible if it is monitored by a tool like ninjarmm which is proprietary.
Until there are some legal precedence we don’t really know how this will be interpreted, but first let’s talk about why this exits: Larger and complex open source projects need revenue sources to keep the developers paid and the product supported and hosting & support services is how Graylog funds this. The SSPL is really a response to Amazon and others who undercut their offerings and do not give back to the projects.
While I hope there are better solutions and business models in the future, for many projects the SSPL is what we have for now.
Maybe it was disabled in the BIOS? Also, if you are installing it in a VM, make sure you either set the CPU Type to “Host” or to another CPU type that supports the AVX instruction set.
I received the same error, however my user is in the correct group. Mine was attempting to write to a mounted drive for the large amount of data. As @LTS_Tom stated in the video… Only I can’t seem to get the owner/group, etc settings correct for the log data to /mnt/data on separate drive.
After running docker-compose up, it runs until I get to;
mongodb exited with code 132
2023-05-21 13:26:37,566 INFO : org.mongodb.driver.cluster - Cluster description not yet available. Waiting for 30000 ms before timing out.
I am following your steps. Not sure why I am gett “Cluster description” since I am not running a cluster.
I am running Ubuntu server 22.04 VM in Proxmox. I am not having any luck searching for a solution to my problem.
The Mongo server requires a CPU with AVX instructions and if your CPU does not have those it will not work. If your CPU does support AVX then make sure they are being passed through in Proxmox. I am not a Proxmox user but based on other posts it does not seem to do that by default.
I ran cat /proc/cpuinfo on host machine and I don’t see AVX in the flags. Computer is old and has an i7 quad core 870@2.93 cpu… Intel’s website says it’s in the Lynnville family.
Now all we need is a volunteer who is willing to test whether Graylog works with these images. On the other hand, most machines without AVX support are already more than 11 years old, so maybe it’s about time for a HW upgrade.
This was just the push I needed to start using docker- i’ve been using graylog on a “dedicated” system for years, and this helped me move in that direction. This was really easy to get up and running, however, i’m having some trouble with persistent data. Is this an appropriate place to post my docker-compose.yml file to get some help, or should I create a new thread?
