Use gemspec executable (the only one or the first specified executable) as default gem entry point for tebako package

[maxirmx]

• gem: check gemspec for executables (but we can only have one..., so the user still needs to choose)

Originally posted by @ronaldtse in #215 (comment)

[bradgessler]

I keep looking at https://github.com/tamatebako/tebako?tab=readme-ov-file#packaging-scenarios-with-ruby and find it mostly confusing, mainly because I don't really know what combination is being used from that table, which leads to surprises.

I think a better approach might be to instead use flags in the press method or have different press commands. For example, I'd like to give tebako a Gemfile and reference a binary in there like this:

ruby "3.3.6"

source "https://rubygems.org"

gem "terminalwire", "~> 0.3.0.alpha"

Then I'd want to run a command like this to generate my executable:

$ tebako press -b Gemfile -e terminalwire-exec

The -o would match the -e by default to keep the command less verbose.

Similarly, perhaps I could pass a gem or gem path into press:

$ tebako press -g terminalwire -v "0.3.0.alpha1" -e terminalwire-exec

Files might work like this:

$ tebako press -b Gemfile -f my-one-off-script.rb -o my-app

This approach would take a lot of the guess work and wondering out of how the binary is being built and probably yield better error messages for misconfigurations.

[maxirmx]

1. Tebako packages directory trees
2. The directory tree has a root
3. In the root of the directory tree there may be:
   Gemfile - none or one
   *.gemspec - none, one or many ("gemspecs")
   *.gem - none, one or many ("gems")
4. Scenarios:
   Error: If the there are many gemspecs it is an error
   Bundled gem: If there is one Gemfile and one gemspec tebako runs

   bundle install
   gem build 
   gem install
   

   Gem: If there is no Gemfile and one gemspec tebako runs

   gem build 
   gem install
   

   Bundle: If there is one Gemfile and no gemspec tebako runs

   bundle install
   

   Packaged gems: If there is no Gemfile and no gemspec but one or many gems tebako runs for each gem

   gem install
   

   Script: If there is no Gemfile, no gemspec and gems tebako just copies files

IMHO there is nothing that is confusing in these rules. One does not need to be Ruby programmer or software engineer to understand them. I do not think that passing \home\maxirmx\project\Gemfile and not \home\maxirmx\project\ as a parameter will change anything.

[maxirmx]

This approach would take a lot of the guess work and wondering out of how the binary is being built and probably yield better error messages for misconfigurations.

Frankly, I do not understand the message above

Here follows a piece of real tebako log. Lines starting with @ are real commands that tebako executes to deploy the solution it packages.

-- Running init script
   ... creating packaging environment at /Users/runner/.tebako/o/s
-- Running deploy script
   ... installing tebako-runtime gem
   ... @ /Users/runner/.tebako/o/s/bin/gem install tebako-runtime --no-document --install-dir /Users/runner/.tebako/o/s/lib/ruby/gems/3.2.0
   ... collecting gem from gemspec /Users/runner/work/tebako-samples/tebako-samples/tutorial/2_packaging_scenarios/gemspec_and_gemfile.sample/tebako-table.gemspec and Gemfile
   ... @ /Users/runner/.tebako/o/s/bin/bundle config set --local build.ffi --disable-system-libffi
   ... @ /Users/runner/.tebako/o/s/bin/bundle config set --local build.nokogiri --no-use-system-libraries
   ... @ /Users/runner/.tebako/o/s/bin/bundle config set --local force_ruby_platform false
   *** It may take a long time for a big project. It takes REALLY long time on Windows ***
   ... @ /Users/runner/.tebako/o/s/bin/bundle install --jobs=3
   ... @ /Users/runner/.tebako/o/s/bin/bundle exec /Users/runner/.tebako/o/s/bin/gem build /Users/runner/work/tebako-samples/tebako-samples/tutorial/2_packaging_scenarios/gemspec_and_gemfile.sample/tebako-table.gemspec
   ... installing /Users/runner/.tebako/o/r/tebako-test-0.0.2.gem gem
   ... @ /Users/runner/.tebako/o/s/bin/gem install /Users/runner/.tebako/o/r/tebako-test-0.0.2.gem --no-document --install-dir /Users/runner/.tebako/o/s/lib/ruby/gems/3.2.0
   ... target entry point will be at /__tebako_memfs__/bin/tebako-table-cli
   ... stripping the output
-- Running mkdwarfs script
   ... @ /Users/runner/.tebako/deps/bin/mkdwarfs -o /Users/runner/.tebako/o/p/fs.bin -i /Users/runner/.tebako/o/s --no-progress

[bradgessler]

Thanks for listing that out! I think I can put together a tebako package Gemfile.exe -e terminalwire-exec command together that:

1. Reads the Ruby Version from the Gemfile.
2. Grab all the gems (and dependencies), and work with path:, github:, etc. and puts them in a folder, probably a bunch of *.gem files.
3. Generates a Tabfile with all the info needed to tebako press with the expected configuration.

This can then be passed into tebako press command to create a package.

IMHO there is nothing that is confusing in these rules. One does not need to be Ruby programmer or software engineer to understand them. I do not think that passing \home\maxirmx\project\Gemfile and not \home\maxirmx\project\ as a parameter will change anything.

It gets confusing when what should be the same commands fails differently on different platforms. I currently have a tebako press command that works on amd64 macOS:

# This works from macOS
$ tebako press -r gem/terminalwire -e terminalwire-exec -R 3.3.6 -o build/macos/amd64/bin/terminalwire-exec
Tebako executable packager version 0.12.1
No prefix specified, using ~/.tebako
Running tebako press at /Users/bradgessler/.tebako
   Mode:                      'bundle'
   Ruby version:              '3.3.6'
   Project root:              '/Users/bradgessler/Projects/terminalwire/ruby/gem/terminalwire'
   Application entry point:   'terminalwire-exec'
   Package file name:         '/Users/bradgessler/Projects/terminalwire/ruby/build/macos/amd64/bin/terminalwire-exec'
   Loging level:              'error'
   Package working directory: '<Host current directory>'
# Lots of log messages ...
[ 94%] Building CXX object CMakeFiles/tebako-fs.dir/src/tebako-main.cpp.o
[ 94%] Building CXX object CMakeFiles/tebako-fs.dir/Users/bradgessler/.tebako/deps/src/tebako/tebako-fs.cpp.o
[100%] Linking CXX static library libtebako-fs.a
[100%] Built target tebako-fs
-- Running finalize script
   ... building tebako package
   ... @ make ruby -j12
   ... @ make -j12
Created tebako package at "/Users/bradgessler/Projects/terminalwire/ruby/build/macos/amd64/bin/terminalwire-exec"
[100%] Built target tebako

But a similar command fails inside an Ubuntu Tebako container:

# This fails from Ubuntu
$ docker run -it -v $PWD:/host tebako-ubuntu-base bash
$(docker-host) tebako press -r /host/gem/terminalwire -e terminalwire-exec -R 3.3.6 -o terminalwire-exec-ubuntu
Tebako executable packager version 0.12.1
Using TEBAKO_PREFIX environment variable as prefix
Running tebako press at /root/.tebako
   Mode:                      'bundle'
   Ruby version:              '3.3.6'
   Project root:              '/host/gem/terminalwire/'
   Application entry point:   'terminalwire-exec'
   Package file name:         '/terminalwire-exec-ubuntu'
   Loging level:              'error'
   Package working directory: '<Host current directory>'
-- Generating files
   ... tebako-version.h
   ... tebako-fs.cpp
   ... deploy.rb
-- Running tebako press script
Running tebako press script
-- OSTYPE: 'linux-gnu'
-- NCORES: 4
Configuration summary:
-- ruby: v3.3.6 at /root/.tebako/deps/src/_ruby_3.3.6
-- dwarfs with tebako wrapper: @v0.9.2 at /root/.tebako/deps/src/_dwarfs_wr
-- DATA_SRC_DIR: /root/.tebako/o/s
-- DATA_PRE_DIR: /root/.tebako/o/r
-- DATA_BIN_DIR: /root/.tebako/o/p
-- DATA_BIN_FILE: /root/.tebako/o/p/fs.bin
-- Target binary directory: /root/.tebako/o/s/bin
-- Target library directory: /root/.tebako/o/s/lib
-- Target local directory: /root/.tebako/o/s/local
-- Target Gem directory:: /root/.tebako/o/s/lib/ruby/gems/3.3.0
-- FS_MOUNT_POINT: /__tebako_memfs__
-- Building for Win32 Ruby (RB_W32): OFF
-- Removing GLIBC_PRIVATE reference: OFF
-- Not building Ruby extensions: dbm,win32,win32ole,-test-/*
-- Ruby build cflags='-fPIC -I/root/.tebako/deps/include -I/usr/local/lib/ruby/gems/3.1.0/gems/tebako-0.12.1/include -fdeclspec'
-- Ruby build LDFLAGS='-L/root/.tebako/deps/lib -L/root/.tebako/o'
-- openssl Ruby option=''
-- libyaml Ruby option=''
-- Configuring done
-- Generating done
-- Build files have been written to: /root/.tebako/o
[ 60%] Built target _patchelf
[ 60%] Built target _dwarfs_wr
[ 89%] Built target _ruby_3.3.6
Tebako setup has been verified
[ 89%] Built target setup
-- Running init script
   ... creating packaging environment at /root/.tebako/o/s
-- Running deploy script
   ... installing tebako-runtime gem
   ... @ /root/.tebako/o/s/bin/gem install tebako-runtime --no-document --install-dir /root/.tebako/o/s/lib/ruby/gems/3.3.0 --bindir /root/.tebako/o/s/bin
   ... collecting gem from gemspec /host/gem/terminalwire/terminalwire.gemspec
   ... @ /root/.tebako/o/s/bin/gem build /host/gem/terminalwire/terminalwire.gemspec
   ... installing /root/.tebako/o/r/terminalwire-0.3.0.alpha1.gem gem
   ... @ /root/.tebako/o/s/bin/gem install /root/.tebako/o/r/terminalwire-0.3.0.alpha1.gem --no-document --install-dir /root/.tebako/o/s/lib/ruby/gems/3.3.0 --bindir /root/.tebako/o/s/bin
   ... target entry point will be at /__tebako_memfs__/bin/terminalwire-exec
tebako-packager failed: Entry point bin/terminalwire-exec does not exist or is not accessible [106]
make[3]: *** [CMakeFiles/packaged_filesystem.dir/build.make:70: CMakeFiles/packaged_filesystem] Error 106
make[2]: *** [CMakeFiles/Makefile2:198: CMakeFiles/packaged_filesystem.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:258: CMakeFiles/tebako.dir/rule] Error 2
make: *** [Makefile:202: tebako] Error 2
Tebako script failed: 'tebako press' build step failed [104]

The log messages make sense, but the error messages don't because I don't understand how tebako arrives at different conclusions about the presence of `bin/terminalwire.

I'm sure I'm doing something stupid or not interpreting the log files correctly, but that's true for most things in software development.

[bradgessler]

I'm sure I'm doing something stupid or not interpreting the log files correctly, but that's true for most things in software development.

I'm usually right when I say this 😂

I've worked out that when tebako builds this gem inside of a container, its running into this, which is the default spec.files provided by gems created with bundler gem:

Gem::Specification.new do |spec|
  # code ...
  spec.files = IO.popen(%w[git ls-files -z], chdir: __dir__, err: IO::NULL) do |ls|
    ls.readlines("\x0", chomp: true).reject do |f|
      (f == gemspec) ||
        f.start_with?(*%w[bin/ test/ spec/ features/ .git .github appveyor Gemfile])
    end
  end
  # code 
end

When I run this from irb inside the container, I get nothing back:

root@0ac7b8e44b97:/host/gem/terminalwire# irb
irb(main):001:0> IO.popen(%w[git ls-files -z], chdir: __dir__, err: IO::NULL).read
=> ""

When I run from the host machine, I get something back:

irb(main):023> IO.popen(%w[git ls-files -z], chdir: __dir__, err: IO::NULL).read
=> ".env\u0000.github/workflows/Dockerfile\u0000.github/workflows/main.yml\u0000.gitignore\u0000.rspec\u0000Brewfile\u0000CHANGELOG.md\u0000CODE_OF_CONDUCT.md\u0000Gemfile\u0000Gemfile.lock\u0000LICENSE.txt\u0000README.md\u0000Rakefile\u0000bin/console\u0000bin/setup\u0000builder/build.sh\u0000builder/template/app/.gitkeep\u0000builder/template/bin/.gitkeep\u0000builder/template/lib/.gitkeep\u0000builder/template/lib/boot.rb\u0000builder/template/lib/boot.sh\u0000builder/template/vendor/.gitkeep\u0000examples/exec/localrails\u0000gem/terminalwire-client/lib/terminalwire-client.rb\u0000gem/terminalwire-client/lib/terminalwire/client.rb\u0000gem/terminalwire-client/lib/terminalwire/client/entitlement.rb\u0000gem/terminalwire-client/lib/terminalwire/client/entitlement/environment_variables.rb\u0000gem/terminalwire-client/lib/terminalwire/client/entitlement/paths.rb\u0000gem/terminalwire-client/lib/terminalwire/client/entitlement/policy.rb\u0000gem/terminalwire-client/lib/terminalwire/client/entitlement/schemes.rb\u0000gem/terminalwire-client/lib/terminalwire/client/exec.rb\u0000gem/terminalwire-client/lib/terminalwire/client/handler.rb\u0000gem/terminalwire-client/lib/terminalwire/client/resource.rb\u0000gem/terminalwire-client/lib/terminalwire/client/server_license_verification.rb\u0000gem/terminalwire-client/spec/entitlement/environment_variables_spec.rb\u0000gem/terminalwire-client/spec/entitlement/paths_spec.rb\u0000gem/terminalwire-client/spec/entitlement/policy_spec.rb\u0000gem/terminalwire-client/spec/entitlement/schemes_spec.rb\u0000gem/terminalwire-client/spec/entitlement_spec.rb\u0000gem/terminalwire-client/spec/resource_spec.rb\u0000gem/terminalwire-client/spec/spec_helper.rb\u0000gem/terminalwire-client/terminalwire-client.gemspec\u0000gem/terminalwire-core/lib/terminalwire-core.rb\u0000gem/terminalwire-core/lib/terminalwire.rb\u0000gem/terminalwire-core/lib/terminalwire/adapter.rb\u0000gem/terminalwire-core/lib/terminalwire/cache.rb\u0000gem/terminalwire-core/lib/terminalwire/logging.rb\u0000gem/terminalwire-core/lib/terminalwire/transport.rb\u0000gem/terminalwire-core/lib/terminalwire/version.rb\u0000gem/terminalwire-core/spec/cache_spec.rb\u0000gem/terminalwire-core/spec/spec_helper.rb\u0000gem/terminalwire-core/spec/terminalwire_spec.rb\u0000gem/terminalwire-core/terminalwire-core.gemspec\u0000gem/terminalwire-rails/lib/generators/terminalwire/install/USAGE\u0000gem/terminalwire-rails/lib/generators/terminalwire/install/install_generator.rb\u0000gem/terminalwire-rails/lib/generators/terminalwire/install/templates/application_terminal.rb.tt\u0000gem/terminalwire-rails/lib/generators/terminalwire/install/templates/bin/terminalwire\u0000gem/terminalwire-rails/lib/generators/terminalwire/install/templates/main_terminal.rb\u0000gem/terminalwire-rails/lib/terminalwire-rails.rb\u0000gem/terminalwire-rails/lib/terminalwire/rails.rb\u0000gem/terminalwire-rails/terminalwire-rails.gemspec\u0000gem/terminalwire-server/lib/terminalwire-server.rb\u0000gem/terminalwire-server/lib/terminalwire/server.rb\u0000gem/terminalwire-server/lib/terminalwire/server/context.rb\u0000gem/terminalwire-server/lib/terminairb(main):024>

I'm pretty sure bundler gem does this to avoid people accidentally releasing gems with files that are not checked into their repos.

[bradgessler]

Running this from the container:

$ git config --global --add safe.directory '*'

And mounting the project root as a volume, with access to the git repo, results in a successful build.

This is a bit brittle though because if the person running press doesn't have their git repo mounted, nothing will build.

[maxirmx]

@bradgessler thank you for reporting this issue

I suggest the following approach

1. Enter container
2. Run tebako scenario commands
   In the case under discussion it will be

gem build ...
gem install ...

3. Run gem executable
4. Run

tebako press ...

5. Run tebako package

If step 2 works and step 3 works but step 4 fails or step 5 fails it is tebako container or tebako issue.

[bradgessler]

I'm confused about this step:

1. Enter container

I searched through the list of tebako commands on the README.md and I'm not sure what it means to "enter container". My mental model for this is based on how Docker works, which means I'd run some sort of command like docker run -it $CONTAINER bash. Obviously Tebako doesn't work that way, but I'm not sure how to build a mental model around this.

Is there a command I can run to "enter [a Tebako] container"? Or do I extract the contents of a tebako package somewhere, do a bunch of things to it, and then somehow package it back up?

Thanks for any pointers on how I'd approach this 🙏

[maxirmx]

This response was generated by ChatGpt

What Does It Mean to "Enter a Docker Container" and How to Do It?

Entering a Docker container means accessing the environment inside the running container. This allows you to execute commands, inspect files, and debug processes directly within the container.

---

Steps to Enter a Docker Container

1. List Running Containers

To identify the container you want to enter, list all currently running containers:

docker ps

This command will display a table of running containers with details such as CONTAINER ID, NAME, and status.

2. Use docker exec to Enter the Container

Once you know the CONTAINER ID or NAME, use the docker exec command to start an interactive shell session inside the container:

docker exec -it <container_id_or_name> /bin/bash
-i: Keeps the input stream open.
-t: Allocates a pseudo-terminal for interaction.

Replace /bin/bash with /bin/sh if the container uses a minimal shell (common in lightweight images like Alpine).
Example:
If the container's name is my_container, use:

docker exec -it my_container /bin/bash

3. Use docker attach (Alternative)

You can also attach directly to the main process of the container:

docker attach <container_id_or_name>

However, this method is less flexible and may inadvertently stop the container if you type exit. It's better suited for monitoring container output.

4. Access a Stopped Container

If the container is not running, start it first:

docker start <container_id_or_name>
docker exec -it <container_id_or_name> /bin/bash

5.Exiting the Container

For docker exec: Type exit to leave the container without stopping it.
For docker attach:
Press Ctrl+P followed by Ctrl+Q to detach without stopping the container.
Typing exit will stop the container.

[bradgessler]

I know how to enter a Docker container, I'm asking "how do I enter a Tebako container", as you stated in this comment:

Did you mean "docker container" and not "tebako container"?

[ronaldtse]

@bradgessler I think @maxirmx was referring to the Tebako CI Containers which can be used to build Tebako packages.

The relevant documentation is here:

• https://github.com/tamatebako/tebako-ci-containers#package-using-a-persistent-container

[ronaldtse]

• gem: check gemspec for executables (but we can only have one..., so the user still needs to choose)

I think it's better if we force the user to define their executable because "choosing the first" is not a good solution. If there is one we can use it. If there are multiple then the user must choose?