Building, Installing, Running... Getting past imagemagick's missing references

[edisonrex]

Hi I am attempting to build this on Ubuntu 20.04, and this happens:

root@Baileys:/home/paul/goswi/goswi-master# go build -tags no_pkgconfig gopkg.in/gographics/imagick.v3/imagick

# gopkg.in/gographics/imagick.v3/imagick

/root/go/pkg/mod/gopkg.in/gographics/imagick.v3@v3.5.0/imagick/affine_matrix.go:9:10: fatal error: MagickWand/MagickWand.h: 
No such file or directory

    9 | #include <MagickWand/MagickWand.h>

      |          ^~~~~~~~~~~~~~~~~~~~~~~~~

compilation terminated.

I can do some env variables to suppress the error(s) in build, but install goes back to the same error. And run doesn't, at all.

I guess it begs the question do you really truly need imagemagick just to convert profile pics? I read all your musings on the reasons for using it, but imagemagick is a disaster of a project to try to build from. Just my opinion though.

I did actually modify the #include to wand/magick_wand.h which is a valid include, but then the build dies on 3 type cast errors.

Have you got this project to build, and how? Is there anything I could help with? I don't know Go at all but I'm pretty good with other things and Go doesn't seem very scary as a language.

Regards

[Tampa]

I got this to work on a fresh install of ubuntu 22 doing the following:

apt-get install golang-go

apt-get install libjpeg-dev libpng-dev libopenjp2-7-dev gcc g++ libltdl-dev

git clone https://github.com/ImageMagick/ImageMagick.git ImageMagick-7.1.1

cd ImageMagick-7.1.1

./configure --with-modules --enable-delegate-build --with-quantum-depth=16 --with-jp2 --with-openjp2=yes --with-jpeg=yes --with-png --with-png=yes

make

make install

apt-get install libmagickwand-dev

cd ..

export CGO_CFLAGS_ALLOW='-Xpreprocessor'

git clone https://github.com/GwynethLlewelyn/goswi.git

cd goswi

cp config.sample.ini config.ini

nano config.ini

nano main.go (Need to fix the ROBUSTserver variable to ROBUSTServer)

go run .

It would be helpful to extend the readme on actually how to start this with go run . as I suspect some might go for just starting main.go and that fails.

Not sure if that imagemagick install that way is even necessary, but I know that's how jp2 support is accomplished with it. Assuming the magickwand hands off to the system install of imagemagick it should work to show the profile images. Though really what you want for this is a separate service that just converts and caches the images somewhere so you can directly serve the png or jpeg files. On the fly conversion is not great as that's really heavy, you want to do that in a controlled way to not overload the system.

With imagemagick installed this way it's fairly trivial to call it and convert the profile and group images or whatever else you want to pull out of assets, but I write php not golang.

The warnings during startup about things not to be used for production or blindly trusting everything is probably not good for long term or production use as the warnings suggest. That's something worth looking into as security, as lackluster as it is with OpenSim, we don't want to make worse.

Seems a lot of effort for just a web panel, not just for setting up imagemagick, but the code in general. I gravitate to php for these things for the ease of use as it doesn't require generating a webserver from scratch. It's, if I see that correctly, a single application that runs as service, so how would it scale? Think rewriting some grid services or an asset server in golang might be a good option, not so much a web panel. Just a personal opinion.

[Usalabs]

There is no full instructions from installing go, to how to configure the config.ini file, to connect to a MySQL database, it seems this application is only for use by the developer, as ONLY the developer knows how to get it working, nobody else knows.

[GwynethLlewelyn]

My apologies for only replying today — I have way too many overlapping things to do, and I confess that this was not very high in my priorities.

1. Aye, 'some assembly required'. Or, rather, 'some Go knowledge required'. In other words: no, I'm not going to teach you how to get a working Go environment to get gOSWI compiled. It's not too difficult, though:

• You can use any package manager, such as apt on Debian/Ubuntu Linux, or homebrew on macOS. Almost all will have a package called go or golang.
• If you're on Ubuntu Linux, I recommend using @longsleep's PPA: https://ppa.launchpadcontent.net/longsleep/golang-backports/ — Simon keeps a much more recent version of Go up to date (from the Debian equivalent) than the 'official' Ubuntu repositories
• If all else fails: Go is surprisingly easy to get and install — have a look at their official installation page

2. The config.ini file configuration instructions are in the supplied config.sample.ini; they should be obvious to anyone who is used to configure OpenSimulator.
3. The connection to the database may not be obvious. It uses a Data Source Name (DSN) which was inspired by what PHP's PEAR::DB uses — more precisely, the 'advanced' format mentioned on the PEAR manual — just dropping the prefix, i.e.: [user[:password]@][net[(addr)]]/dbname[?param1=value1&paramN=valueN] (brackets indicate optional parameters; the parentheses are required to connect to a remote host and/or a local socket which is not the default one)
   I've added that comment to the config.sample.ini file as well.
4. Finally, the source code is provided 'as-is', which essentially means that there isn't any free support. Caveat utilitor.

Now, I'm aware that the major source of trouble is to get a working, functional copy of ImageMagick 7 on Debian/Ubuntu. Even though ImageMagick 7 has been released nine years ago (!), it still remains unsupported on those distros. However, there is a pre-compiled package for an upcoming Debian release (still 'under testing') which does work under Ubuntu as well: it has 'only' twenty (!!) dependencies that must be additionally downloaded, but then Ubuntu's apt will pretty much figure out what it additionally needs and pull those from its usual package repos.

In order to show how this works in practice, look at the Git Actions for this repository, which will build, from scratch, a working Go + ImageMagick 7 in order to compile gOSWI. Please note that, every time a Git Actions workflow is run, it starts with an empty, pristine Ubuntu installation and begins to install everything else. This means that within a certain timeframe (since everything will be continuously upgraded!), you can rest assured that it is possible to have everything properly configured to get it working.

The magic happens here — where you can see which packages have been added — and of course, by clicking on the Action itself, you can check what has actually happened and replicate it on your own system. Take into account that this will only work on Ubuntu 24.04 or an equivalent recent Debian/Ubuntu derivative distro.

(Note: macOS users have it much easier, since, at the time of writing, Homebrew supports ImageMagick 7.1.1-41 directly).

That said...

Indeed, this is a 'personal project' which was created with two main goals: allowing my grid users to have a quick way to change their passwords — because they keep forgetting them — and providing a 'splash screen' with a list of regions to directly teleport to, as well as a working map. Very secondarily, it also provides some statistics (mostly for statistic aggregators) and little more than that.

Since I don't have a huge grid to work with, a more distributed solution was not really required. Go apps are pretty good at launching gazillions of lightweight threads in parallel to deal with whatever is thrown at them; the question is mostly if you wish to deploy a distributed system to ease the load and to make the single point of access a bit more reliable. In this scenario, and assuming a transparent backend with its own built-in redundancy checking (namely, the MySQL database), there is nothing strictly forbidding to deploy several instances of gOSWI running simultaneously — assuming that the front-end load balancing system is able to deal with cookies (i.e., guaranteeing that the same HTTP stream is kept alive for the same user, until they log out or there is a network disconnect). A minimalistic approach, using DNS to provide round-robin distribution of load, and possibly having nginx acting as a distributed-load front-end server, as described by documents such as these or these — I would personally recommend one of those hash-based solutions, but, again, YMMV.

I agree that 'this is just a webpanel'. It was never meant to be anything else! I'm also fond of PHP, of course, and I can agree that for 'just a webpanel', doing everything in PHP would have been easier. Or — perhaps not. I would very likely need to use one of those massive frameworks, such as Laravel or Symphony — both of which are challenging in their own way. Go has its own 'framework' built into its standard libraries; and I use Gin Gonic on top of that to make everything even easier to work with. In other words — there is not much work to do except grab some HTML/CSS and paste it on a handful of templates, and voilà, you have a full-blown web application in an instant, without really needing to 'learn' the complexities of one of the 'popular' PHP frameworks.

Regarding JPEG2000 and the usage of ImageMagick 7 for converting between formats... with that, I cannot disagree with your proposals, you're quite right on everything you mentioned: it's like killing a fly with an atomic bomb, since ImageMagick 7 is immensely complex and there is no port to Go, just a layer on top of the C/C++ libraries (and even that is a small nightmare to manage), which needs a 'special' feature of Go (known as Cgo) that allows Go to make calls to a C/C++ library 'as if' it were written in Go natively. The problem is that this bit of code needs to be compiled with a C/C++ compiler, and that is an additional layer of complexity of its own. Generally speaking, Go will have no problem in launching whatever C/C++ compiler is available and do a reasonably good job at making the integration run smoothly. But that, in turn, requires struggling with specific dependencies just to get it to compile — and my 'quick & dirty' approach is shown on the go.yml workflow.

That said, I guess that, for future versions, I might simply resort to forking a magick process instead — it should also work that way, at the cost of a slight performance hit (spawning/forking whole processes has way more overhead than the built-in, lightweight 'goroutines'). Then again, unlike PHP, this is all compiled to native machine code, and should certainly have as much performance as you can expect from it.

[mteedev]

You are amazing! Thank you for you!