NOTE: These fixes are mainly intended to facilitate downstream
integration. libjpeg-turbo is not officially supported with any of
these macros undefined. (That is, it is not guaranteed to work. It is
expected that downstream integrators diagnose and report any issues that
may occur. Those issues will not be fixed proactively.)
- bmpsizetest.c: Fix build if C_LOSSLESS_SUPPORTED or
D_LOSSLESS_SUPPORTED is undefined.
- cjpeg.c, jpegtran.c: Fix compiler warnings if
C_MULTISCAN_FILES_SUPPORTED or C_PROGRESSIVE_SUPPORTED is undefined.
- jcdctmgr.c, jcinit.c, jddctmgr.c, jdmaster.c, simdcoverage.c: Fix
build if all DCT_*_SUPPORTED macros are undefined. (This allows for
building libjpeg-turbo with only lossless JPEG support.)
- jclhuff.c, jcmaster.c: Fix build if ENTROPY_OPT_SUPPORTED is
undefined.
- jcmaster.c: Fix compiler warning if C_PROGRESSIVE_SUPPORTED is
undefined.
- jcsample.c: Fix compiler warnings if INPUT_SMOOTHING_SUPPORTED is
undefined.
- jdapistd.c: Fix build if UPSAMPLE_MERGING_SUPPORTED is undefined.
- jmorecfg.h:
* Clarify that C_PROGRESSIVE_SUPPORTED requires
C_MULTISCAN_FILES_SUPPORTED and ENTROPY_OPT_SUPPORTED. (The
progressive Huffman encoder won't even build without the latter.)
* Clarify that D_PROGRESSIVE_SUPPORTED and D_LOSSLESS_SUPPORTED
require D_MULTISCAN_FILES_SUPPORTED. (The progressive huffman
decoder and difference controller won't even build without it.)
* Clarify that IDCT_SCALING_SUPPORTED requires DCT_ISLOW_SUPPORTED.
* Remove vestigial UPSAMPLE_SCALING_SUPPORTED macro. (I guess this
was intended for a feature that was never implemented.)
- jpegtran.c: Fix compiler warning is TRANSFORMS_SUPPORTED is 0.
- turbojpeg.c:
* Do not advertise any scaling factors other than {1, 1} if
IDCT_SCALING_SUPPORTED is undefined. Otherwise, calling
applications may erroneously expect a scaled image when an unscaled
image will be provided, which may cause them to provide too small of
a buffer.
* Fix build if SAVE_MARKERS_SUPPORTED is undefined.
* Fix build if TRANSFORMS_SUPPORTED is 0.
- turbojpeg-mp.c: Fix compiler warning if C_LOSSLESS_SUPPORTED is
undefined.
Most RVV implementations issue instructions based on LMUL (the vector
length multiplier), even if VL (the vector length) is less than VLMAX
(the maximum number of elements that will fit in the register.) Thus,
even though our DCT/IDCT implementations always operate on 8 elements at
a time (VL=8), we can improve performance with RVV implementations that
have 256-bit or wider registers (VLEN>=256) by explicitly using
half-register (LMUL=1/2) intrinsics.
Note that LMUL=1/2 with 64-bit elements (SEW=64) isn't allowed, so the
8x8 transpose algorithm has to temporarily use full-register intrinsics.
- Optimize range limiting in the IDCT algorithms.
- Implement an in-register matrix transpose in the IDCT algorithms
rather than using strided stores/loads to/from memory.
- Avoid non-contiguous memory accesses in the fancy upsampling
algorithms.
Emscripten has a built-in SIMD intrinsics translator that works with our
Neon SIMD extensions. Compared to plain WASM, this approach has mixed
performance results at the algorithm level and is slower overall.
Arm granted explicit permission.
Per 076e5f1cae, the remaining Neon code
that I didn't author was submitted in #628 without "All Rights
Reserved", and I added the phrase for consistency. Now I'm removing the
phrase for consistency.
Based on:
d48f1ff2fb137fe9ef726d2f276fd0462954dc51d5ac20d3ef6c68dcaad4cdb255c60facb51a1d8db5b660b629
Improvements relative to the aforementioned implementation:
- ~145% faster RGB-to-YCbCr color conversion
~156% faster grayscale-to-YCbCr color conversion
~94% faster h2v1 downsampling
~167% faster h2v2 downsampling
~159% faster h2v2 fancy upsampling
~202% faster h2v1 plain upsampling
~290% faster h2v2 plain upsampling
~64% faster YCbCr-to-RGB color conversion
~54% faster h2v1 merged upsampling
~59% faster h2v2 merged upsampling
This was mostly achieved by:
* reducing non-contiguous memory accesses (including replacing strided
loads/stores with segmented loads/stores),
* using RVV instructions to generate a bias vector for downsampling,
rather than using a clunky dynamically-allocated array and C code,
and
* restructuring the algorithms (usually based on my experience with
other SIMD module implementations) to improve throughput and reduce
register pressure.
The DCT algorithms are unfortunately suboptimal at the moment, because
they use strided loads/stores to transpose the 8x8 coefficient matrix.
I tried using segmented loads/stores, as well as implementing an
in-register transpose with *vrgather*(). The former did not improve
performance significantly, and the latter was slower. The new Zvzip
RVV extension should make it possible to implement a more efficient
in-register transpose, which may open up opportunities for further
optimization. (The color conversion algorithms may benefit from that
as well.)
- ~10-26% (average 18%) better overall RGB-to-baseline compression
performance, and ~1-14% (average 8%) better overall baseline-to-RGB
decompression performance. (The forward and inverse DCT algorithms
are still suboptimal, and they dominate overall performance.)
- Improved build-time detection of compiler support for RVV intrinsics
- Support for run-time RVV detection on *BSD (hypothetically, at least--
I can only verify that the code compiles properly)
- Cleaned up run-time RVV detection on Linux
- Cleaned up SIMD dispatcher integration
- Descriptive code comments
- Corrected and added copyright attributions
- Code formatting tweaks
Closes#620Closes#837
- "All Rights Reserved" is no longer necessary, since all Buenos Aires
Convention signatories signed onto the Berne Convention in 2000.
- The phrase is confusing with respect to open source software, since
all rights are not in fact reserved. (The IJG README file even
clarifies: "All Rights Reserved except as specified below.")
- The phrase is even more confusing when a piece of source code has
multiple authors, because in that case, no one person owns the
copyright. (Thus, no one person can reserve all of the rights.)
- The zlib-licensed SIMD source code used the phrase inconsistently.
Historically, I introduced the phrase in conjunction with the
BSD-licensed TurboJPEG API source code in 1.2.x, probably because I
saw the phrase used in other BSD-licensed software. It then crept
into the Neon SIMD extensions in 1.2.x, the DSPr2 SIMD extensions in
1.4.x, the AltiVec SIMD extensions in 1.5.x, and the MMI SIMD
extensions in 2.0.x, due to similar oversights. However, because the
x86 SIMD extensions never used the phrase, the SIMD dispatchers (which
were based upon the x86 SIMD dispatcher) never used it either.
- Other libjpeg-turbo contributors who added "All Rights Reserved" to
their copyright notices did so only to conform to the existing
convention of the libjpeg-turbo code base, so this commit does not
violate their intent, nor does it change the legal meaning or strength
of their attributions.
- The Modified (3-clause) BSD License and zlib License never contained
the phrase "All Rights Reserved" (although earlier BSD licenses did),
so the addition of the phrase to our licenses could be incorrectly
interpreted as a license variation.
There are plenty of examples of open source software that uses "All
Rights Reserved" in conjunction with the BSD or zlib License. I was
probably following one of those examples when I used the phrase in my
own code. Using "All Rights Reserved" was encouraged for individual
copyright holders prior to the year 2000, so those of us who started
creating content in the 1990s or before still fall back upon that old
habit sometimes.
With respect to code I didn't author, the only thing that might be
construed to legally forbid this change is the first clause in the
Modified BSD License, which reads:
- Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
and the last clause in the zlib License, which reads:
3. This notice may not be removed or altered from any source
distribution.
However, the spirit of these clauses is to prevent downstream users and
distributors from altering authorship attributions and thus implying
that they wrote the code. The clauses were not intended to prevent
upstream maintainers from removing a legally obsolete and confusing
phrase from contributed code that they committed, particularly when the
phrase (a) was never meant to be used with the license in the first
place, and (b) was, in most cases, added by the maintainer rather than
the contributor.
The contributions this applies to are from:
- https://sourceforge.net/p/libjpeg-turbo/patches/74
- #158
- #538
- #845
In these cases, the code was submitted without a copyright attribution,
and I added it or requested that it be added on behalf of the authors
(who expressed their intent for the attribution to conform to the code
base's existing copyright header format.)
- #628
In this case, the code was submitted without "All Rights Reserved"
attached, and I added the phrase for consistency.
- https://sourceforge.net/p/libjpeg-turbo/mailman/message/32981447
In this case, I obtained explicit permission from the author to remove
the phrase.
- #455
Permission has been requested, so this change has not yet been applied
to the Neon SIMD extensions.
- Put a space after (C) in BSD-licensed source files.
- Add missing trailing period in jcphuff-sse2.asm.
- Remove comma after copyright date in src/md5/md5.c.
libjpeg and our SIMD extensions referred to "pixels", "pixel rows", and
"pixel values" in the comments for the upsampling and downsampling
algorithms. "Pixel" describes a group of components that are co-located
in the spatial domain, regardless of how they are arranged in memory
(e.g. packed or planar.) Thus, you can technically refer to a set of
spatially co-located components in separate planes as a "pixel."
However, it is more clear to describe the downsampling input and
upsampling output as "components", since not all components from the
original pixels are necessarily downsampled or upsampled.
If -maltivec is passed to the compiler, then __ALTIVEC__ will be
defined, so we need not include any run-time detection headers or
variables on any platform.
The NULL color conversion, H2V2 smooth downsampling, 6x6 integer inverse
DCT, 12x12 integer inverse DCT, and integer upsampling algorithms were
only ever implemented for DSPr2, and it's unclear whether those
implementations were optimal. There may eventually be other SIMD
implementations of the 6x6 and 12x12 integer inverse DCT algorithms, but
SIMD implementations of the other aforementioned algorithms are unlikely
and discouraged. (NULL color conversion was never a very good candidate
for SIMD acceleration, since it is primarily limited by memory
performance, and smooth downsampling and integer upsampling are so
rarely used that they are not worth accelerating.)
It is our convention to name floating point sample
conversion/quantization implementations jquantf* and integer sample
conversion/quantization implementations jquanti*.
Use the algorithm/module names from simd/README.md in the file headers
of the corresponding SIMD module implementations, as well as in the
SIMD API headers.
Referring to #835, there were long-standing bugs in some of the modules,
which necessitated disabling those modules. Over the past four months,
I have tried and failed to find any parties interested in fixing those
bugs (including the parties who contributed the DSPr2 extensions.) I
interpret that as a strong sign that no one in the community cares about
the DSPr2 extensions anymore, but let's see if anyone complains before
3.2 beta1.
See ChangeLog.md for additional justifications, but suffice it to say
that:
- MIPS was a dead architecture in 2021.
- In 2026, it's shuffled off its mortal coil, run down the curtain, and
joined the bleedin' choir invisible.
Following up on ced602ad2f,
- Rename simd/jsimd.h to simd/jsimdint.h to avoid confusion and to be
somewhat consistent with the header naming conventions of the libjpeg
API library.
- Eliminate unnecessary #include directives in the SIMD modules.
- Move src/jsimd.h and src/jsimddct.h to simd/. This makes the code
easier to understand, since jsimd.h and jsimddct.h define the external
interfaces for libsimd. This also ensures that all SIMD-related code
and zlib-licensed code lives in the same directory.
- In jsimd.h and jsimddct.h, remove references to libjpeg/SIMD
and jsimdext.inc and replace them with the complete zlib License
header. jsimdext.inc only exists in the x86 SIMD extensions, and
jsimd.h and jsimddct.h never contained any code derived from
libjpeg/SIMD.
- Guard all inclusions of jsimd.h and jsimddct.h in the libjpeg API
library source code with #ifdef WITH_SIMD. This eliminates the need
for those headers to have knowledge of the macros defined in
jconfig*.h.
- Fix the generation of jsimdcfg.inc (broken by
ced602ad2fbe3a37f8c0266dd2c4909a3f64a2b6.)
As with other architectures, there is no need to autodetect the
availability of a particular SIMD instruction set if the compiler
targets an architecture that requires that instruction set. If the code
is built with -mloongson-mmi, then the resulting binary will not work on
CPUs that lack MMI instructions, so libjpeg-turbo can safely assume that
MMI instructions are available.
More specifically, using /proc/cpuinfo to autodetect SIMD instruction
set availability is unreliable under QEMU, and QEMU is the only
mechanism by which I can test the correctness of the DSPr2 and MMI
SIMD extensions.
If compiler warnings are treated as errors, then the apostrophe in
"Loongson MMI can't work with MIPS Release 6+" triggers a compiler
warning that causes the HAVE_MMI test to fail. Surrounding the error
message with quotes fixes the issue.
- Use a new preprocessor macro (SIMD_ARCHITECTURE), set by the build
system, to ensure that the same implementation of the SIMD extensions
is selected at configure time and compile time. This eliminates any
dependence on compiler-specific and CPU-specific preprocessor macros.
- Where possible, consolidate all CPU-specific SIMD dispatcher functions
into a common source file (simd/jsimd.c) with CPU-specific code paths
selected by the aforementioned SIMD_ARCHITECTURE preprocessor macro.
- Implement common wrappers for the SIMD module functions that take
different arguments than their C counterparts. (Some SIMD modules do
that because it is difficult to interface with jpeg_*compress_struct
in assembly language.) For such modules, store the SIMD module
function pointers in the opaque jpeg_color_converter,
jpeg_downsampler, jpeg_entropy_encoder, jpeg_inverse_dct,
jpeg_upsampler, and jpeg_color_deconverter structures.
- Eliminate unnecessary wrappers for the SIMD module functions that take
the same arguments as their C counterparts. The dispatchers for those
modules now reuse the existing function pointers in the opaque
my_fdct_controller and phuff_entropy_encoder structures.
- Streamline the implementation of both the C and SIMD extended RGB
color conversion functions. The source and destination colorspaces
are locked in when jpeg_start_*compress() is called, so we can choose
an extended RGB color conversion function at that time. It isn't
necessary to wait until color conversion actually occurs.
- Initialize the SIMD extensions on a per-instance basis rather than a
per-thread basis. This eliminates any dependence on thread-local
storage in the libjpeg API library. (Referring to #834, TLS can be
problematic with older Windows versions.) The TurboJPEG API library
still relies on TLS when throwing errors from global functions, but
that is a less critical feature than the SIMD extensions.
This also makes the initialization of the SIMD extensions more
intuitive, predictable, and controllable from the point of view of
calling programs. (For AArch32, MIPS, and PowerPC architectures,
initializing the SIMD extensions on Linux might require reading
/proc/cpuinfo, which likely incurs a bit of overhead. Thus, it's good
to know exactly when that will occur.)
- Implement architecture-specific jpeg_simd_cpu_support() functions for
all non-x86 architectures, and move non-x86 CPU feature detection code
into the new functions.
- When using GCC or Clang, mark all SIMD dispatcher and module functions
with __attribute__((visibility("hidden"))) so that they will not be
exposed in the symbol table of shared libraries. In practice, the
version script/mapfile already accomplishes this on ELF platforms,
and Windows hides global symbols by default. Thus, this primarily
benefits macOS.
- Set the default value of the SIMD support variable to the bitwise
inverse of all possible bitmask values. This makes it impossible to
mistake an uninitialized value for an initialized/supported value.
- Eliminate the JSIMD_FORCEAVX2 environment variable. It had the effect
of enabling only the AVX2 SIMD modules if AVX2 instructions were
supported. That isn't particularly useful, since not all modules have
an AVX2 implementation.
- Enable SSE SIMD modules with JSIMD_FORCESSE2. This makes the behavior
of JSIMD_FORCESSE2 consistent with that of JSIMD_FORCESSE and
JSIMD_FORCE3DNOW, i.e. "Emulate the behavior of the library when
running on a CPU that supports only this instruction set and older
ones."
- Introduce a new test program (simdcoverage) that reports the SIMD
modules that will be used. This facilitates diagnosing dispatcher
issues.
- Clean up SIMD #includes; ensure that all SIMD headers include their
own dependencies.
- Fix various strict compiler warnings in the SIMD code.
- Eliminate unnecessary preprocessor checks in SIMD dispatcher
functions. (For instance, DCTSIZE will never not be 8.)
- In the non-x86 SIMD extensions, remove references to libjpeg/SIMD and
jsimdext.inc and replace them with the complete zlib License header.
jsimdext.inc only exists in the x86 SIMD extensions, as does any code
derived from libjpeg/SIMD.
These were a relic of libjpeg/SIMD, which attempted to follow the
conventions of the libjpeg source code, but they are no longer relevant
(or even accurate in some cases.)
