SBT Native Packager Building RPM with Python files

I was working on a project where Airflow Python code was packaged as part of RPMs. By default, the rpmbuild process compiles Python files, which Airflow does not accept — it expects to compile them itself. So the RPM’s job was to ship the files raw, without compilation.

Here is the complete runsheet for building RPM files using SBT Native Packager while disabling Java jar recompression and Python compilation.

Setup

Edit project/plugins.sbt and add the plugin:

addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.1.1")

Disable Java Jar Repackaging and Python Compilation

Add the following file at src/main/rpm/pre.

The RPM pre-scriptlet executes just before the package is installed. Read more about RPM scripts at this link.

%global __os_install_post %(echo '%{__os_install_post}' | sed -e 's!/usr/lib[^[:space:]]*/brp-java-repack-jars[[:space:]].*$!!g' | sed -e 's!/usr/lib[^[:space:]]*/brp-python-bytecompile[[:space:]].*$!!g')

This script removes the brp-java-repack-jars and brp-python-bytecompile processes from the RPM post-install step.

To make the pre file work, add the following to your build configuration:

import RpmConstants._
maintainerScripts in Rpm := maintainerScriptsAppendFromFile((maintainerScripts in Rpm).value)(
  Pre -> (sourceDirectory.value / "main" / "rpm" / "pre")
)

Complete rpm.sbt

import sbt.Keys._
import NativePackagerHelper._

enablePlugins(UniversalPlugin)
enablePlugins(RpmPlugin, RpmDeployPlugin)

maintainer in Linux := "admin@mail.com"
packageSummary in Linux := "Package code"
packageDescription := s"Application ${name.value}"
rpmVendor := "My company"

version in Rpm 
  v.trim.replace("-SNAPSHOT", s".${System.currentTimeMillis}")
}

rpmLicense := Option("Apache")
rpmObsoletes := Seq(s"${name.value}")
// This does not work
rpmBrpJavaRepackJars := false
defaultLinuxInstallLocation := "/var/lib/airflow"
// This allows to override the install location using RPM prefix
rpmPrefix := Some(defaultLinuxInstallLocation.value)
import RpmConstants._

// Remove all jars
mappings in Universal := (mappings in Universal).value.filterNot{
case (file, fname) => fname.endsWith(".jar")}

// Add fat jar
mappings in Universal += {
  val fatJar = (assembly in Compile).value
  fatJar -> s"${name.value}.jar"
}

// Copy contents of dags folder
mappings in Universal ++= directory("src/main/dags")

// Add version.txt file
mappings in Universal += {
  val file = target.value / "version.txt"
  IO.write(file, s"${(version in Rpm).value}")
  file -> "version.txt"
}

import RpmConstants._
maintainerScripts in Rpm := maintainerScriptsAppendFromFile(
(maintainerScripts in Rpm).value)(
  Pre -> (sourceDirectory.value / "main" / "rpm" / "pre")
)

publishTo 
  val artifactory = "https://artifactory.com/"
  if (v.trim.endsWith("SNAPSHOT"))
    Some("Artifactory Realm" at artifactory + "artifactory/rpm-snapshots")
  else
    Some("Artifactory Realm" at artifactory + "artifactory/rpm-releases/")
}

Build and Publish

Build the RPM:

rpm:packageBin

Publish the RPM:

rpm:publish

References

• RPM Spec Reference: Scripts