1

I am excited to announce AppBundler.jl :partying_face:

The package offers recipes for building Julia GUI applications in modern desktop application installer formats. It uses Snap for Linux, MSIX for Windows, and DMG for MacOS as targets. It bundles full Julia within the app, which, together with artifact caching stored in scratch space, allows the bundling to be done quickly, significantly shortening the feedback loop.

The build product of AppBundler.jl is a bundle that can be conveniently finalised with a shell script on the corresponding host system without an internet connection. This allows me to avoid maintaining multiple Julia installations for different hosts and reduces failures due to a misconfigured system state. It is ideal for a Virtualbox setup where the bundle together with bundling script is sent over SSH after which the finalised installer is retrieved.

The configuration options for each installer bundle vary greatly and are virtually limitless; thus, creating a single bundling configuration file for all systems is impractical. To resolve this, the AppBundler recipe system comes into the picture. AppBundler provides default configuration files which substitute a few set variables specified at the Project.toml in a dedicated [bundle] section. This shall cover plenty of use cases. In case the application needs more control, like interfacing web camera, speaker, host network server, etc., the user can place a custom snap.yaml, AppxManifest.xml and Entitlements.plist in the application meta folder, overloading the defaults. Additional files can be provided easily for the bundle by placing them in a corresponding folder hierarchy. For instance, this can be useful for providing custom-sized icon sizes. To see how that works, explore AppBundler.jl/examples and PeaceFounderClient where you can check out the releases page to see what one can expect.

All recipes define a USER_DATA environment variable where apps can store their data. On Linux and Windows those are designated application locations which get removed with the uninstallation of the app, whereas on MacOS, apps use ~/.config/myapp and ~/.cache/myapp folders unless one manages to get an app running from a sandbox in which case the $HOME/Library/Application Support/Local folder will be used.

Thought has also been put into improving the precompilation experience to reduce start-up time for the first run. For MacOS, precompilation can be done before bundling in the /Applications folder by running MyApp.app/Contents/MacOS/precompile. For Linux, precompilation is hooked into the snap configure hook executed after installation. For Windows, a splash screen is shown during the first run, providing user feedback that something is happening. Hopefully, the cache relocability fix in Julia 1.11 will allow us to precompile the Windows bundle as well.

The most challenging aspect of this package is crafting recipes. So far, I have yet to manage to get sandboxing to work on either platform which prevents applications from being accepted in corresponding marketplaces. In particular, the issues are:

  • Windows can start Julia and crash immediately. See issue #52007
  • MacOS can start GUI but is unresponsive. Present in both GTK and QML.
  • Linux can start GLWF by linking with the mesa-core22 snap content package. GTK apps with mesa-core22 segfaults, but works if unlinked. QML draws a window but does not render content #191

It would probably be reasonable to integrate PackageCompiler.jl for completeness and to resolve the sandboxing issues with Windows and MacOS in that way. One MacOS application is still standing in the Mac app store developed with now deprecated ApplicationBuilder.jl so the path is clear. However, at the moment, I am only considering investing time in it once PackageCompiler issue #164 is resolved, as there would be no benefit for the PeaceFounder project.

Another avenue worth exploring could be making a GitHub workflow, but making a Windows build setup seems like a pain in the ass. At the moment, I am doing postprocessing with shell scripts for MacOS and Windows.

33 Likes
2

I’m guessing this doesn’t handle edge-cases such as when user-dirs.dirs exists, XDG_CACHE_HOME is set, or someone has run SHSetKnownFolderPath on Windows?

I ask because these sort of edge cases motivated me to make GitHub - tecosaur/BaseDirs.jl: A cross platform implementation of the XDG Directory Spec.

4 Likes
3

This is good. I did not understand what the XDG environment variables were all about, but now, reading the specs, it makes sense. Your package looks like something which should work within the GUI application. Still, it needs to include the cases for situations where it is run from a snap environment, which has its own data directories SNAP_USER_DATA and SNAP_USER_COMMON where XDG directories are often pointed to. It is trickier for UWP, which does not provide an environment variable where user data is stored. I had to infer from the installation folder name and process it to get its location in the $LOCALAPPDATA/Packages directory.

2 Likes
4

This is pretty great, thank you for creating it! Is flatpak or appimage support on the horizon? Asking as I have the impression these are more widely supported than snap on linux.

2 Likes
5

Currently, my focus is making Snap support for the Linux platform better, and I have no plans to add Flatpak support as that would duplicate efforts. One of the key advantages Snap offers over Flatpak is the ability to create bundles entirely on other platforms using the widely available mksquashfs utility, which is also built with Yggdrasil. This contrasts with Flatpak, where flatpak-builder is necessary, and manual implementation would require considerably more effort. Depending on it would result in a finalisation step similar to MSIX bundles on Windows. In contrast, Snaps, coupled with a configure script that handles precompilation post-installation are complete and do not need any additional post-processing steps.

Another point worth noting is that Snaps are a good format for distributing server-side applications, which aligns well with the needs of my project. Therefore, while Flatpak has its merits, Snaps are easier to build and are more versatile.

2 Likes
6

It sounds like what might work best for you then is handling the special Snap/UWP cases yourself, and then using BaseDirs.jl for the rest (e.g. config/cache).

Oh, you could also manually do something like

if haskey(ENV, "SNAP_USER_DATA")
    ENV["XDG_DATA_HOME"] = ENV["SNAP_USER_DATA"]
end

before loading BaseDirs (or just call BaseDirs.reload() afterwards) and handle your special cases that way.

1 Like
7

This is supercool, as one of the limitations of Julia is that there was not an easy way to distribute double-clickable apps. Thanks for doing this!

Who know, maybe in the future this can be integrated into VScode, making creating apps as easy as it was in CodeWarrior!

3 Likes
8

Thanks, this is superfast to “build”, as expected, since it’s just bundling, not compiling (it’s not either or, both could be done theoretically, or partially compiling), first time slower if downloading stuff…, and also compressing a bit slower, though not very:

julia> @time AppBundler.bundle_app(Linux(:x86_64), ".julia/packages/AppBundler/bFhGI/examples/qmlapp/", "build/MyApp3-x64")
[ Info: Rule with origin linux/meta is skipped as not found in default or override path.
  Activating project at `/tmp/temp_env`
  Activating project at `~/.julia/environments/v1.10`
  5.528476 seconds (839.69 k allocations: 83.795 MiB, 0.33% gc time)

The docs seem very extensive and good, though I may look into improving minor points.

I’m trying out all the (GUI) examples, and as you can see they are rather large as expected:

$ du -hs build/*
696M	build/MyApp2-x64
1017M	build/MyApp3-x64
216M	build/MyApp4-x64
963M	build/MyApp-x64

MyApp4-x64 is the QML one, but note redone from the 5x larger uncompressed one, MyApp3-x64, done with (as in docs … except):

julia> AppBundler.bundle_app(Linux(:x86_64), ".julia/packages/AppBundler/bFhGI/examples/qmlapp/", "build/MyApp4-x64", compress=true)

I kind of expect the mousetrap example to be the smallest one, but it’s can’t be built since it doesn’t have a Manifest file, and I don’t know how to make one, only now needing it! I reported it and promised a PR when I figured it out, and he doesn’t have time now, so also if others want to do it or explain, even point to docs on that, which is elusive to find.

I’ll try to make an even smaller example, try a text-based “hello world” I expect should work, and may try it on the recent platform Julia game (with the Julia game engine).

I can run the apps, by locating the code in the folders (will finish, figure out one file-installation), but at least the one-file compressed is not the way to go for that:

shell> chmod u+x ./build/MyApp4-x64

shell> ./build/MyApp4-x64
/bin/bash: line 1: ./build/MyApp4-x64: cannot execute binary file: Exec format error

shell> file ./build/MyApp4-x64
./build/MyApp4-x64: Squashfs filesystem, little endian, version 4.0, xz compressed, 226076129 bytes, 21347 inodes, blocksize: 131072 bytes, created: Thu Jan 18 13:02:12 2024
9

I wrote a blog post reflecting on the future of AppBundler :dizzy:

On the potential of integrating bundling toolchains, which would make it easier to write CI deployment actions and integration tests. I discuss current issues in resolving sandboxing issues for MSIX and Snap and some further ideas about making Julia distributions for environments that take a lot of time to precompile.

I am also writing an NLNET grant application and would appreciate feedback, which is linked at the bottom of the blog post.

4 Likes
10

AppBundler v0.2

AppBundler.jl can now create DMG installers for Julia applications using only open-source tools. This new version eliminates host system dependencies by distributing everything as binary JLL components.

I’ve focused on making the bundling process work reliably across Julia-supported POSIX systems without requiring additional software installation. This approach makes AppBundler much easier to test and deploy in various environments.

The new DMG functionality is simple to use through the AppBundler.build_dmg function:

import AppBundler
import Pkg.BinaryPlatforms: MacOS

# Create a .app bundle
AppBundler.build_app(MacOS(:x86_64), "MyApp", "build/MyApp.app")

# Create a .dmg installer with automatic LZMA compression
AppBundler.build_app(MacOS(:x86_64), "MyApp", "build/MyApp.dmg")

These commands work with any example from the AppBundler.jl/examples directory. Your project only needs a main.jl file and a Manifest.toml file so dependencies can be properly tracked.

By default, AppBundler precompiles your application and checks compatibility with the target system. You can set precompile=false to build MacOS applications on Linux and FreeBSD, with precompilation happening on first launch instead. In the future, Julia’s cross-compilation capabilities may allow building installers for all platforms from a single Linux host.

For application signing, you can provide your own certificate in meta/macos/certificate.pfx, protected by a password set in the MACOS_PFX_PASSWORD environment variable. Without a custom certificate, AppBundler generates a self-signing certificate automatically, making the process more accessible for testing.

All code signing is validated in CI with codesign -v MyApp.app to ensure compatibility with standard GUI applications. Note that recent MacOS updates require additional steps to open self-signed applications, involving the Settings/Privacy & Security section.

For advanced users building non-Julia applications or requiring custom layouts, the AppBundler.DMGPack.pac2dmgfunction handles code signing and bundling using rcodesign, xorriso, libdmg_hfsplus, and ds_store utilities.

The next development phase will focus on similar MSIX packaging for Windows applications using open-source tools. I’ve already made progress repurposing makemsix and osslsigncode, though fixing the CMake project for makemsix to use Yggradsil-built dependencies remains a challenge.

Once MSIX support is complete, we’ll develop CI integration pipelines that automatically compile applications for each platform and bundle them as native installers upon GitHub release tagging. This will showcase Julia’s potential for GUI application development.

More details are available in our recent pull request: Replace MacOS SDK with cross-platform open-source alternatives for DMG creation by JanisErdmanis · Pull Request #16 · PeaceFounder/AppBundler.jl · GitHub

18 Likes
11

AppBundler v0.3.0

A new version of AppBundler has been released with integrated MSIX bundling functionality. As with DMG creation, it eliminates any reliance on host system dependencies by distributing everything as binary JLL components.

The MSIX bundling functionality works on all Julia supported platforms, including Windows, which was a significant challenge to make makemsix compile with the MinGW compiler.

The MSIX bundling can be tried on QMLApp through the following commands:

using AppBundler
using Pkg.BinaryPlatforms: Windows

qmlapp = joinpath(dirname(dirname(pathof(AppBundler))), "examples/qmlapp")
destination = joinpath(homedir(), "Desktop/qmlapp.msix")

AppBundler.build_app(Windows(:x86_64), qmlapp, destination)

By default, AppBundler precompiles your application and checks compatibility with the target system. You can set precompile=false to build MSIX on non-Windows systems where the precompilation would then be deferred to first application launch instead.

For application signing, you can provide your own certificate in meta/windows/certificate.pfx protected by a password set in the WINDOWS_PFX_PASSWORD environment variable. Without a custom certificate, AppBundler generates a self-signing certificate automatically, making the process more accessible for testing, just like with DMG creation.

For advanced uses like building non-Julia applications or requiring custom layouts, use AppBundler.MSIXPack.pack2msix. A higher-level function AppBundler.build_msix is also available that sets up MSIX directory structure and allows users to customize the application staging area. A similar function will be added for DMG in the next release.

The next step of development will focus on refactoring the missing pieces: using the julia/etc/julia/startup.jl also for Snap packaging, implementing AppBundler.build_snap and AppBundler.build_dmg in a similar way, cleaning up some tests, fixing documentation, and slashing unneeded dependencies.

The current state now also allows setting up CI integration pipelines that automatically compile applications for each platform and bundle them as native installers upon GitHub release tagging. This will make cross-platform desktop application deployment a much more pleasant experience.

More details are available in the recent pull request: Implementing MSIX packaging via makemsix and osslsigncode by JanisErdmanis · Pull Request #17 · PeaceFounder/AppBundler.jl · GitHub

15 Likes
12

I just want to chime in and say that it’s really cool to see how much work you’re putting in here! It seems particularly timely to me with the push to make static compilation possible :heart_eyes:.

I’ve got one bit of feedback from a look at the project. It looks like you’ve picked Snaps as the sole linux packaging format. While it may be a pain to add more options, I think that most Linux users would appreciate having options besides Snap: Flatpack is installed by default on more distros, AppImage “just works” everywhere, and .rpms and .deps don’t seem to be going anywhere soon. The ugly truth is that Linux is far from settling on a supported-everywhere packaging solution.

I’m not sure what the most valuable additions would be, or what the packaging process would be like for Flatpack/AppImage, but options would be nice. FWIW, since Julia doesn’t link against system stuff much (other than libc et al.) I suspect that .rpm and .dep archives would actually be fairly easy to put together.

I expect there are time/expertise constraints here. Perhaps this could be a good thing to advertise being interested in contributions for?

2 Likes
13

Thank you for your kind words :hugs: I am also eager to see static compilation to happen.

Concerning the Linux packaging, it seems the world has settled on Snap and Flatpak, as they offer sandboxing capabilities that enhance users’ security. Unfortunately, when I examined how to create a Flatpak, I encountered the requirement of using a Flatpak builder. Even then, it remained unclear how to produce a standalone Flatpak that is not bound to the concept of a repository. I would appreciate help here to figure this out.

1 Like
14

I’m curious what would be included under “non-Julia applications?” Maybe along the same lines, as I understand it, AppBundler assumes a main.jl so that the main executable in the resulting bundle is initiated from Julia code. Is there a way to bundle an app in which the main is a C (or other C ABI-compatible precompiled executable for each platform) that is dynamically bound to the julia runtime (via libjulia.dll dynamic embedding) and loads Julia code from scripts (not precompiled to enable “cross-compilation”). We would want to ship these with any required packages that are known before packaging, as well as related platform-specific binaries, but allow for dynamic download in some cases at the deployment location. Is this sort of scenario supported in an AppBundler configuration?

15

It seems you could use the build_msix function:

AppBundler.build_msix(srcdir, "myapp.msix") do staging_area
    # Compile, or place application files  
    # Use PackageCompiler or bundle_app function
end

In this situation, the srcdir only expects Project.toml, from which some general metadata is extracted. The srcdir/meta is optional for setting overrides. In this scenario, one would likely want to set the srcdir/meta/AppxManifest.xml override to point to a custom application launcher.

The bundle_app function handles the platform-dependent binaries, using Manifest.toml to bundle the environment dependencies, including platform-dependent binary artefacts. The override julia/etc/julia/startup.jl is created such that the depot path overlay is pointed to USER_DATA, where dependencies can be dynamically downloaded at the deployment location during application use.

In case of using libjulia.dll you could run julia/etc/julia/startup.jl manually to set up the environment and then run the necessary scripts.

16

Excellent! I’ll give this a try when my project is a bit further along.