1
0
mirror of https://github.com/jqlang/jq.git synced 2025-04-18 17:24:01 +03:00

1796 Commits

Author SHA1 Message Date
Emanuele Torre
aea8efaf0b
jv_unique: don't leak non-unique elements that are not returned (#3304) 2025-03-29 07:16:46 +09:00
dependabot[bot]
947fcbbb1f
build(deps): bump jsonschema from 4.18.4 to 4.23.0 in /docs (#3302)
Bumps [jsonschema](https://github.com/python-jsonschema/jsonschema) from 4.18.4 to 4.23.0.
- [Release notes](https://github.com/python-jsonschema/jsonschema/releases)
- [Changelog](https://github.com/python-jsonschema/jsonschema/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/python-jsonschema/jsonschema/compare/v4.18.4...v4.23.0)

---
updated-dependencies:
- dependency-name: jsonschema
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-28 09:57:51 +09:00
dependabot[bot]
1e22960cf5
build(deps): bump lxml from 4.9.3 to 5.3.1 in /docs (#3301)
Bumps [lxml](https://github.com/lxml/lxml) from 4.9.3 to 5.3.1.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.9.3...lxml-5.3.1)

---
updated-dependencies:
- dependency-name: lxml
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-28 09:56:51 +09:00
dependabot[bot]
0b2bda8108
build(deps): bump markdown from 3.4.3 to 3.7 in /docs (#3300)
Bumps [markdown](https://github.com/Python-Markdown/markdown) from 3.4.3 to 3.7.
- [Release notes](https://github.com/Python-Markdown/markdown/releases)
- [Changelog](https://github.com/Python-Markdown/markdown/blob/master/docs/changelog.md)
- [Commits](https://github.com/Python-Markdown/markdown/compare/3.4.3...3.7)

---
updated-dependencies:
- dependency-name: markdown
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-28 09:55:39 +09:00
itchyny
5f7b1aed16
fix usage in help to use jq not the invoked command path (#3299) 2025-03-28 08:43:06 +09:00
itchyny
8819eb8fb4
Enable dependabot PR to update Pipenv dependencies (#3270) 2025-03-28 08:42:44 +09:00
dependabot[bot]
313b0e3735
build(deps): bump jinja2 from 3.1.5 to 3.1.6 in /docs (#3298)
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.5...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-27 21:33:55 +09:00
liviubobocu
4b84a6e2f6
Include column number in parser and compiler error messages (#3257)
Resolves #1027, resolves #2334, and resolves #3256.
2025-03-10 22:17:49 +09:00
itchyny
668871bdfa
Fix warning: sign-compare in @base64d format (#3287) 2025-03-06 17:07:54 +09:00
itchyny
94fd973ebb
Fix calloc error on @base64d format (ref #3280) (#3286)
This commit fixes a regression of a8ce2ff, e.g. `"=" | @base64d`.
2025-03-06 04:30:46 +09:00
Nicole Wren
c70ae1ada4
fix: foreach should not break init backtracking with DUPN (#3266)
Simplifies the compilation of `foreach` to avoid the problem in #3227 by
removing unnecessary `FORK ... BACKTRACK`.

Does not address `reduce`, which is less trivial.
2025-03-05 22:01:48 +09:00
sachint
f65d088b75
Enable IBM z/OS support (#3277) 2025-03-05 17:21:05 +09:00
itchyny
7e5cad359a
Add autotools temporary files to .gitignore (#3285) 2025-03-05 16:57:11 +09:00
SArpnt
9d6efec198
fix: unset environment variables before shtests (#3283)
Tests no longer fail if `JQ_COLORS` and `NO_COLOR` are already set.
2025-03-05 07:59:46 +09:00
itchyny
a09a4dfd55 Reject NaN with payload while parsing JSON
This commit drops support for parsing NaN with payload in JSON like
`NaN123` and fixes CVE-2024-53427. Other JSON extensions like `NaN` and
`Infinity` are still supported. Fixes #3023, fixes #3196, fixes #3246.
2025-03-05 07:43:54 +09:00
itchyny
a8ce2ff09a
Avoid zero-length calloc (#3280)
Using `calloc` function with a count of zero elements is implementation
defined and it may cause allocation errors on some platforms (ref #3277).
This commit fixes `jv_mem_calloc` callers to avoid undefined behavior.
2025-03-04 22:33:51 +09:00
itchyny
4003202ccf
Fix segmentation fault on strftime/1 and strflocaltime/1 (#3271)
When we use a large format string to `strftime/1` and `strflocaltime/1`,
we get segmentation fault. This happens because the result buffer is
allocated on the stack using `alloca` function, and `strftime` function
writes out of the space when the size is not enough to format time.
I fixed the segmentation fault issue by allocating the result buffer on
the heap using `malloc` function, because `alloca` function does not
provide a way to detect insufficient space.
2025-03-04 22:28:09 +09:00
itchyny
dc849e9bb7
Improve performance of repeating strings (#3272)
This commit improves the performance of repeating strings, by copying
the result string instead of the string being repeated. Also it adds
an error message when the result string is too long.
2025-03-04 22:13:55 +09:00
itchyny
5df471fa5b
Disable Valgrind by default during testing (#3269)
Previously, the configure script automatically enables Valgrind during
testing when the valgrind command is found. However, running tests with
Valgrind is quite slow, so it is better to disable it by default, and
to enable it only when the user specifies the `--enable-valgrind` flag.
2025-03-04 22:12:25 +09:00
itchyny
c4e1fb08fb
Support CRLF line breaks in filters (#3274) 2025-03-04 22:11:40 +09:00
itchyny
b7a1201b72
Preserve numerical precision on length/0 (#3275) 2025-03-04 22:09:33 +09:00
itchyny
17be97a8ad
Define _BSD_SOURCE in builtin.c for OpenBSD support (#3278)
On OpenBSD, some prototypes are hidden behind `_BSD_SOURCE`.
Defining this variable fixes #3252, and also fixes #3276.
This can be seen as a regression of #2631.
2025-03-04 21:39:52 +09:00
itchyny
95ee663b28
improve performance of unique and unique_by (#3254)
Previously, `unique` and `unique_by` filters are implemented using
`group_by` and then `map(.[0])`. This commit re-implements in C, to
avoid unnecessary boxing of grouping, and improves the performance.
2025-03-04 19:59:41 +09:00
Owen Ou
8ba03f788f
Replace jqplay.org with play.jqlang.org (#3265)
As discussed in https://github.com/jqlang/jq/discussions/3244, we are
moving jqplay to the jqlang GitHub organization. This updates all
references from jqplay.org to play.jqlang.org.
2025-02-18 08:22:05 +09:00
Marcin Serwin
f3d4c1165d
fix: copy the string returned by getenv (#3264) 2025-02-18 07:54:13 +09:00
itchyny
70e72abea3
Fix compiler warning type-limits in found_string (ref #2911) (#3263)
Whether `char` has a sign is implementation dependent, and `c >= 0` may
result in a compiler warning. We can use bitwise operator to check
whether the character is a control character, regardless of the sign.
2025-02-18 07:52:58 +09:00
Marcin Serwin
6d8e9f98ae
fix: populate timezone data when formatting time (#3203)
The `jv2tm` function was zeroing fields of `struct tm` that were not
specified by the standard. However, depending on the libc this produced
incorrect timezone data when used together with formatting functions.
This change tries to fill the timezone data using either `mktime`,
`timegm`, or manually.

Apple's Libc implementation contains a bug which causes it to ignore
the offset data present in the `struct tm` in favor of the older heuristic
needed by legacy standards. This workaround temporarily sets the global
timezone so it gets picked up during formatting.
2025-02-17 21:41:01 +09:00
itchyny
b86ff49f46
fix: jv_number_value should cache the double value of literal numbers (#3245)
The code of `jv_number_value` is intended to cache the double value of
literal numbers, but it does not work because it accepts the `jv` struct
by value. This patch fixes the behavior by checking if the double value
is `NaN`, which indicates the unconverted value. This patch improves the
performance of major use cases; e.g. `range(1000000)` runs 25% faster.
2025-02-16 22:08:36 +09:00
itchyny
64bd8ba374
fix: jv_dump_string_trunc should not break unicode characters (#3249) 2025-02-16 21:40:18 +09:00
itchyny
fa6a2ff688
website: remove unused root path on building website (#3248)
We no longer use the `--root` option of `build_website.py` since we have
moved to jqlang.org (from jqlang.github.io/jq where the root was `/jq`).
We don't need the option on local development either since `http.server`
has the `-d` option to specify the directory to serve from.
2025-02-06 22:42:27 +09:00
Mattias Wadman
e33578a53e
Add opengraph meta tags (#3247)
Makes the site preview a bit nicer when sharing
2025-02-06 22:14:29 +09:00
itchyny
6ce3bdf382
fix: preserve numerical precision on unary negation (#3242) 2025-02-06 07:50:21 +09:00
itchyny
972772153f
Move oniguruma and decNumber to vendor directory (#3234) 2025-02-06 07:49:56 +09:00
Mattias Wadman
2755664a8e
Non-matched optional capture group should be null and offset -1 (#3238)
Fixes #3093
2025-02-04 19:18:58 +01:00
itchyny
d0adcbfebb
Fix --indent 0 implicitly enabling compact-output (#3232)
Now it will retain pretty-printing, just remove any indentation.

Co-authored-by: Andrew Marshall <andrew@johnandrewmarshall.com>
Co-authored-by: Gabriel Marin <marin.gabriel@protonmail.com>
2025-02-01 15:09:30 +09:00
orbea
07af9c11b7
build: fix insecure RUNPATH (#3212)
The -static-libtool-libs flag causes GNU libtool to add a rpath to the .libs
build directory, which is intended for internal use by the libtool
implementation and is not installed in the resulting package causing Gentoo to
print a QA notice. See Gentoo issue: https://bugs.gentoo.org/945698.
2025-02-01 15:07:54 +09:00
itchyny
0d1e371e63
website: switch to custom domain jqlang.org (#3243) 2025-02-01 11:54:32 +09:00
itchyny
dea5e61cde
docs: fix example links to jqplay.org (#3241) 2025-01-28 22:15:19 +09:00
itchyny
ce54621904
docs: mention the behavior of scan using regex with capturing groups (#3240) 2025-01-28 11:41:17 +09:00
itchyny
39eea145b4
website: deploy website from GitHub Actions (#3237)
Deploying GitHub Pages from the gh-pages branch is a classic way.
We can use actions/deploy-pages action to deploy GitHub Pages.
2025-01-28 11:40:55 +09:00
Han Jiang
da28628edd
Fix redundant subexpression in Generators and iterators example in doc (#3239) 2025-01-27 19:45:27 +09:00
itchyny
31dac287cc
tests: quote .[] in shtest which can break syntax highlight (#3235) 2025-01-26 08:08:31 +09:00
Eloy Coto
562ffec869
refactor: move bsearch function to C-code (#2945)
This commit fixes issue #527 and move the bsearch function to a native
C-code. This change improves the performance of this function.
2025-01-25 22:09:43 +09:00
Michael Färber
18bd8374c1
Escape some code parts in manual. (#3184) 2025-01-25 21:55:06 +09:00
Dag-Erling Smørgrav
d8463c2549
Move closer to the standard autotools idiom. (#3187)
* Create `src/config.h` instead of passing everything on the compiler
  command line.

* To reduce the magnitude of this change, add `--include src/config.h`
  to CFLAGS instead of adding `#include "src/config.h"` at the top of
  each source file.  Not all compilers support this, but I don't think
  we care about anything other than gcc and clang at this point.

* Rather than generate `src/config_opts.inc`, emit `JQ_CONFIG` directly
  into `src/config.h`.

* Rather than generate `src/version.h`, define `JQ_VERSION` as an alias
  for the standard `PACKAGE_VERSION`.
2025-01-25 17:10:05 +09:00
itchyny
71ed7b3412
tests: refactor constant folding tests (#3233) 2025-01-25 13:58:47 +09:00
dependabot[bot]
588ff1874c
build(deps): bump jinja2 from 3.1.4 to 3.1.5 in /docs (#3226)
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.5.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.4...3.1.5)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-29 21:54:10 +09:00
lectrical
bcbf2b4616
Generate provenance attestations for release artifacts and docker image (#3225)
Adding https://github.com/actions/attest-build-provenance to the ci builds so
that the release assets and docker image for the next release tag generate
signed build provenance attestations for workflow artifacts.
2024-12-29 21:53:16 +09:00
Emanuele Torre
8bcdc9304a jq_next: simplify CALL_BUILTIN implementation 2024-12-01 11:49:44 +01:00
Emanuele Torre
0b82b3841b builtin.c: typecheck builtin cfunctions in function_list
In C23 (default C standard used by GCC 15),  jv (*fptr)();  has become
equivalent to  jv (*fptr)(void);  so we can no longer assign builtin
implemenations directly to the fptr member of cfunctions without
generating a compile error.

Since there does not seem to be any straight-forward way to tell
autoconf to force the compiler to use C99 short of explicitly adding
-std=c99 to CFLAGS, it is probably a cleaner solution to just make the
code C23 compatible.

A possible solution could have been to just redeclare  cfunction.fptr
as void*, but then the functions' return type would not have been type
checked (e.g. if you tried to add a {printf, "printf", 2}, where printf
is a function that does not return jv, the compiler wouldn't have
complained.)
We were already not typechecking the arguments of the functions, so e.g.
  {binop_plus, "_plus", 3},  /* instead of {f_plus, "_plus, 3},       */
  {f_setpath, "setpath", 4}, /* instead of {f_setpath, "setpath", 3}, */
compile without errors despite not having the correct prototype.

So I thought of instead improving the situation by redefining
cfunction.fptr as a union of function pointers with the prototypes that
the jq bytecode interpreter can call, and use a macro to add the builtin
functions to function_list using to the arity argument to assign the
implementation function to the appropriate union member.

Now the code won't compile if the wrong arity, or an arity not supported
by the bytecode interpreter (>5 = 1input+4arguments), or a prototype not
jallable by the bytecode interpreter (e.g. binop_plus that doesn't
expect a  jq_state*  argument).

Also, the code now compiles with gcc -std=c23.

Fixes #3206
2024-12-01 11:49:44 +01:00