Java’s ServiceLoader API + using native libraries => NOK

The problem!

I tried to build Apache Parquet on my ARM64 machine but it failed with:

$ mvn clean verify
...
Tests in error:
testReadWriteWithCountDeprecated(org.apache.parquet.hadoop.DeprecatedInputFormatTest): org.apache.hadoop.io.compress.CompressionCodec: Provider org.apache.hadoop.io.compress.BrotliCodec could not be instantiated
private static final ServiceLoader<CompressionCodec> CODEC_PROVIDERS = ServiceLoader.load(CompressionCodec.class);

The workaround!

At the time of writing this article (20.02.2021, Apache Parquet 1.11.1) the only reliable solution is to remove manually brotli-codec.jar from the classpath.

<dependencies>
...
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-hadoop</artifactId>
<version>1.11.1</version> <!-- or later version -->
<exclusions>
<exclusion>
<groupId>com.github.rdblue</groupId>
<artifactId>brotli-codec</artifactId>
</exclusion> </exclusions> </dependency>
...
</dependencies>

The solution!

I’ve sent a Pull Request to https://github.com/rdblue/brotli-codec migrating the Java Brotli bindings from jbrotli to Brotli4j because it is maintained and provides native binaries for several more CPU architectures. But Ryan Blue is no more interested in maintaining his library and he would prefer a new Brotli codec to be introduced.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store