Add Linux installer

[jasp00]

This allows a Debian-based host to build an installer with makeself, running make makeself. It should take up about 29 MiB. This should address #1620 and #2556, in the sense that a Linux binary is provided, not Debian or Ubuntu packages.

I have added a script that fetches the source for bundled libraries in the Linux installer; this takes care of #2937 partly. After building the installer, run make bundled-source. There is a lot to download.

[tresf]

Linux users... @Umcaruje @mikobuntu @grejppi what are you opinions on a home-directory install versus system-wide?

[Umcaruje]

I still don't get why we couldn't just offer downloads of .deb and .rpm packages. I'm just used to installing software with a .deb or by using a PPA. The home directory install sounds just confusing to me.

[tresf]

We can, but .deb files don't come bundled with dependencies and generally require a package manager to fulfill them which varies between distros so we'll likely find ourselves offering various debs for different Debian-based distros. I'd assume OpenSUSE/SUSE and Fedora/RHEL/Centos suffer the same problems from an .rpm perspective. I believe this is part of the problem people have with the KXStudio repos (work, but bundle some libraries people already have) although I don't know the specifics, I've just seen people on IRC complaining about the DEBs from there.

I've supported desktop software on Linux before and the .run files work very well for what we're doing. They're essentially acting just like the dmg and exe files do, but for Linux.

[midi-pascal]

I like the .run idea too, as the Qt standalone installer works: if you run it as a "normal" user it installs in your home directory and if you run it as root it installs in /opt. The best of the two worlds...

[tresf]

👍

[tresf]

@midi-pascal does qt install to ~/.qt like @jasp00 is proposing?

[midi-pascal]

@tresf Yep!

[mikobuntu]

@tres I'm all for a self contained ~home directory install. This is usually how I will install most software that I build myself. I guess most users will not be running a machine with multiple users, therefore a system wide install is not needed anyway.

[tresf]

This is a great start. I've left some comments in the script for consideration. At a glance I think it could use some commenting to explain what's happening. Some of the DEB stuff isn't clear to me at a glance and could really benefit from some commenting.

I also would like some more feedback on the idea of a home-directory install versus something like system-wide /opt/. I'd like some feedback on that from Linux users as it can directly affect the mimetype stuff.

[teeberg]

A home-directory install would be more portable. When I do backups or update/reinstall my system, the first thing I include is my (almost) whole home directory. Including it there would make it easier to move around, if that's something you guys are concerned about :)

You may also potentially not even have write permissions to /opt/, e.g. on a machine with multiple users. I wouldn't assume for the user to have root access and it'd sure be nice to not require them to install. Although, of course, if you intend for this to be installed via a package manager, that'll probably require root access anyway, so may not be something to worry about.

[tresf]

A home-directory install would be more portable. When I do backups or update/reinstall my system, the first thing I include is my (almost) whole home directory. Including it there would make it easier to move around, if that's something you guys are concerned about :)

Hmm... Projects are already stored with this in mind and they use relative paths for the samples and plugins so as long as the LMMS home directory is set properly.

"Moving around" executables with projects isn't a concept I'm familiar with. I personally use LMMS with 5 others and we share samples, projects, presets, SF2s, VSTs on Linux, Mac and PC (mac not having capabilities to play VSTs or SF2s yet of course per #698, #649) and we have no issues. No issues migrating to a new system and no issues migrating to a different OS. Well, at least if we're all running the same version, which is currently a challenge when relying on 3rd party repositories on the various Linux distributions as they tend to be behind.

You may also potentially not even have write permissions to /opt/, e.g. on a machine with multiple users. I wouldn't assume for the user to have root access and it'd sure be nice to not require them to install. Although, of course, if you intend for this to be installed via a package manager, that'll probably require root access anyway, so may not be something to worry about.

Nearly every software package on every major Desktop OS requires admin, so AFAIK this can be assumed. There are always exceptions (Putty on Windows, PortableFirefox... DMGs on Apple technically can be installed anywhere because they're portable by design -- they require a drag-and-drop into a folder -- but 99% of the time go to a root-owned location /Applications/) but I rarely see this in Linux so I'm with @Umcaruje at least from the perspective that it sounds confusing.

Although in this case we can "have our cake and eat it to". Makeself's can have parameters passed in, so we can absolutely support both.

Any subsequent arguments to the archive will be passed as additional arguments to the embedded command. You should explicitly use the -- special command- line construct before any such options to make sure that Makeself will not try to interpret them.

But if we do support a user-space install, I still vote that it's the non-standard behavior and that a root-style shared installer is priority. This ensures any user of the workstation can share the application and also have his/her .mmpz files automatically associated system-wide. It's what users are used to, it's what companies are used to (there are schools which deploy LMMS to shared workstations).

P.S. Thanks for feedback. 👍

[tresf]

Actually, so as long as we can detect if it's running as root, perhaps we just display a warning if not running as root and support both out of the box.

[jasp00]

System-wide installation is done when running as root. To build the installer, run make makeself -C cmake/linux. fakeroot is required.

[jasp00]

This new version keeps copyright files. This should probably be done with all the installers.

[tresf]

fakeroot is required.

Can you please explain expand a bit on this requirement?

[zapashcanon]

See this link.

[tresf]

See this link

I know what fakeroot is, I'm curious as to the specific need.

[jasp00]

I have added a script that fetches the source for bundled libraries in the Linux installer; this takes care of #2937 partly. After building the installer, run make bundled-source -C cmake/linux; there is a lot to download.

fakeroot is required.

To ease system-wide installation, umask is not restricted on extract. As a side effect, the temporary directory is not restricted and others than root may access it. Root extracts using owner information from the tarball. Therefore, a user that matches the owner could modify the installation files. To prevent this, the tarball is created with root ownership.

[tresf]

So you've added the complexity of fakeroot for the offchance a person peeks at the temp files during extraction? Why?

[tresf]

Ok I've read up on the owner/tmp vulnerabilities but makeself switched to unmask 077 back in 2011, shouldn't that cover it?

[jasp00]

The vulnerability could mean arbitrary code run by root. fakeroot is a cheap solution. I would have used tar's --owner 0 instead if the released version of makeself accepted the argument.

[jasp00]

makeself switched to unmask 077 back in 2011, shouldn't that cover it?

Yes, but that umask restriction makes a system-wide installation a bit more complicated, because the files need to get their permissions back.

[jasp00]

You may see that I have disabled the umask restriction.

[tresf]

Yes, but that umask restriction makes a system-wide installation a bit more complicated, because the files need to get their permissions back.

Are you sure? I'm still confused. I use makeself all the time. AFAIK it caches the current umask for use later. I see your umask hack is version dependent, but I just don't comprehend what the issue is. I accept that it's most likely the right solution, but I fail to see the reason and I've never had to install fakeroot on a build machine for packaging, so it seems odd.

[jasp00]

AFAIK it caches the current umask for use later.

An installer made with current unreleased makeself does cache umask; one with version 2.2.0 does not. The latter is the one included in Trusty (Travis).

[jasp00]

With the umask set to 077, the system-wide installation is only usable by root. Assuming the original umask is 022, the installer script should run chmod -R go+rX *.

[tresf]

the installer script should run chmod -R go+rX *

Exactly. That's what my other trusty installers do. Let's use that technique instead of fakeroot.

[tresf]

FYI, the update-desktop-database command will fail to work with makeself unless you temporarily shim in a more permissive umask.

[jasp00]

Let's use that technique instead of fakeroot.

Done. I have added support for update-desktop-database as well.

[jasp00]

You may run make makeself in the top directory now. apt-get download is used now, since aptitude does not detect errors.

[tresf]

Still no luck.

Observations:

• Orphaned symlink ld-linux.so.2 -> ld-2.19.so. Copied from /lib/i386-linux-gnu/ld-2.19.so to no avail.

Upgrading a bunch of packages, will try again momentarily.

(gdb) run
Starting program: /opt/lmms-1.1.90-4e9090d/usr/bin/lmms 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
[New Thread 0xb0489b40 (LWP 25306)]
*** Error in `/opt/lmms-1.1.90-4e9090d/usr/bin/lmms': munmap_chunk(): invalid pointer: 0x083663c8 ***

Program received signal SIGSEGV, Segmentation fault.
0xb0619ee7 in ?? () from /usr/lib/i386-linux-gnu/libxkbcommon-x11.so.0
(gdb) 

[tresf]

Updating packages fixed it. Seems to be in good shape on 14.04. 👍

sudo apt-get upgrade
sudo reboot

mkdir build && cd build && cmake .. -DWANT_QT5=True && make -j4 makeself
sudo ./lmms-1.1.90-4e9090d-i386-linux-gnu.run
# [...]
rm ~/.lmmsrc.xml
/opt/lmms-1.1.90-4e9090d/bin/lmms-launcher

[jasp00]

Now dangling symlinks are dropped.

I guess your libxkbcommon-x11-0 package was not up-to-date. Anyway, upgrading the system does not fix the problem, but hides it. Bundled libxkbcommon0 should not use system libxkbcommon-x11-0; this optional package needs to be bundled.

[jasp00]

I thought libxkbcommon-x11-0 was optional, but an updated libqt5gui5 depends on it. I see that libxkbcommon0 is not bundled in the tree above, perhaps because the package was not downloaded and the error was not detected. Although an upgraded system hides these problems, an upgraded system is necessary to fetch the right packages for the installer.

[jasp00]

Bundled libsoundio1 and libgig6 were not used; this is fixed. On a system based on Debian Squeeze, there are some start errors from iconv_open, Fontconfig, GLib and GTK+, but the program works without noticeable issues. There may be problems with libraries that use RPATH, such as PulseAudio; this can be solved with chrpath.

[jasp00]

There may be problems with libraries that use RPATH

Done. If there are no more comments, I will merge this and start the integration with Travis.

[tresf]

here may be problems with libraries that use RPATH, such as PulseAudio; this can be solved with chrpath. [...] Done.

Great. We already use the Mac-equivalent tool for a similar purpose. Note, chrpath isn't part of any of our build tutorials, it should be added to the build packaging instructions above.

I have a 32-bit installer made with Ubuntu 14.04 + Qt5 here: https://github.com/tresf/lmms/releases/download/v1.2.0-makeself/lmms-1.1.90-432a04b-i386-linux-gnu.run

I'm going to test this on a few Linux OSs and post my results.

[tresf]

@jasp00 so I'm doing some portability testing and I'm trying to run the 32-bit installer on a 64-bit OS. I'm not sure if this is recommended or not.

• On Crunchbang 11 x64 I can't seem to fulfill the harfbuzz dependency. I could not find a workaround for this. Edit: Tried make makeself on 14.04 x64.... perhaps the fetch is having a problem.

• On Fedora Core 22 x64, I was able to fulfill the first round of dependencies with harfbuzz.i686, mesa-libGL.i686 but now it complains:

  [fedora@localhost ~]$ /opt/lmms-foo/bin/lmms-launcher 
  This application failed to start because it could not find or load the Qt platform plugin "xcb".
  
  Reinstalling the application may fix this problem.
  /opt/lmms-foo/bin/lmms-launcher: line 5:  9730 Aborted
               (core dumped) "$DIR"/usr/bin/ld.so "$DIR"/usr/bin/lmms "$@"
  

  Which I'm confused about because libqxcb.so is located at /opt/lmms-foo/usr/lib/i386-linux-gnu/qt5/plugins/platforms/libqxcb.so

[tresf]

Update: I tried the 64-bit installer; It seems to suffer the same launch issues on Fedora Core 22 x64 as the 32-bit installer did.

[fedora@localhost ~]$ sudo rm -rf /opt/lmms-1.1.90-432a04b/
[fedora@localhost ~]$ sudo sh lmms-1.1.90-432a04b-x86_64-linux-gnu.run
Verifying archive integrity... All good.
Uncompressing LMMS  100%  
Installing to /opt/lmms-1.1.90-432a04b
[fedora@localhost ~]$ /opt/lmms-1.1.90-432a04b/bin/lmms-launcher 
This application failed to start because it could not find or load the Qt platform plugin "xcb".

Reinstalling the application may fix this problem.
/opt/lmms-1.1.90-432a04b/bin/lmms-launcher: line 5: 11222 Aborted
                 (core dumped) "$DIR"/usr/bin/ld.so "$DIR"/usr/bin/lmms "$@"

[jasp00]

I'm trying to run the 32-bit installer on a 64-bit OS.

I did that test, it should work.

[jasp00]

Which I'm confused about because libqxcb.so is located at /opt/lmms-foo/usr/lib/i386-linux-gnu/qt5/plugins/platforms/libqxcb.so

I guess Qt 5 is looking under /usr instead of under /opt/lmms-foo/usr, which means Qt 5 needs to be patched. If we are not going to switch the Linux build to Qt 5 in the short term, is it worth to maintain the patched library now?

[tresf]

If we are not going to switch the Linux build to Qt 5 in the short term, is it worth to maintain the patched library now?

I would say this is absolutely worth patching now as AFAIK, the only thing that's broken with Qt5 is VSTs GUI (#2855) so once that issue is resolved, we'd have to patch this anyway (right?)

[simonvanderveldt]

I know it's a bit late, but doesn't it make more sense to use AppImage (or maybe flatpak, but that's Wayland only afaik) for this?

[tresf]

@simonvanderveldt, here's the discussion that preceded this PR: #2556 (comment)

[probonopd]

I know it's a bit late, but doesn't it make more sense to use AppImage (or maybe flatpak, but that's Wayland only afaik) for this?

It will be trivially easy to convert a makeself-based installer to an AppImage. So why not have both.

[tresf]

Superseded by #3688. If a demand for a dedicated .run installer arises, we can add a new target to cmake and adjust the packaging script to re-use the linuxdeployqt utility, thus fixing issues like #2932 (comment).

[EndrII]

@tresf maybe you can try cqtdeployer. this tool can create a exe, run, and zip and deb packages from yur compiled application.

[tresf]

@tresf maybe you can try cqtdeployer. this tool can create a exe, run, and zip and deb packages from yur compiled application.

Thanks for sharing, this tool looks promising. At this time, we've settled upon using .AppImage. If .run is brought back, we'll take a look at your utility.