Using stack_p wasn't a portable concept in 12.3 when JSON
parsing got unlimited depth. To let ST_GeomFromGeoJSON was
already a recursive function, needed because object order of "type"
may be after the "geometries", but with json_engine_t no longer
enforcing the depth, some stack checking was required.
Use the check_stack_depth function to allow excessively deep
GeoJSON objects to error.
As this is cleaned up the gis-json test can be enabled.
The exceeding stack depth is moved to lotofstack.test.
ST_GEOMFROMGEOJSON requires JSON keys to appear in a specific order
(type before geometries/features/geometry/coordinates). When keys
appear in a different order, the JSON scanner is left in an
inconsistent state because json_skip_level() is not called to advance
past the value when the type has not yet been determined.
Fix: add json_skip_level() calls in the geometries, features, and
geometry key handlers when the type key has not yet been encountered.
This mirrors the existing pattern in the coordinates handler.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Geometry::create_from_json used strings as things that reinitialize
the je (json_engine). By reinitializing with json_start the killed
pointer gets reset, and also the concept of depth is reset.
There are two states in create_from_json:
1. The search if for the type, once found ci is set
2. Search for the argument, arg becomes != T_NONE
If the type is found first, ci become the class, and once
the arg is found, continued execution on the same je occurs.
If the argument comes before the "type", we've save a copy
of the json_engine_t to process this argument.
In both cases je after the execution is the most progressed
pointer.
Debug assertions on je state aren't enforced by code, if
some invalid GeoJSON is passed, these are actual errors.
Because Geometry::create_from_json is recursive, from
geometrymcollections, a json_read_value is pushed from inside this
function up to Item_func_geometry_from_json::val_str.
Other Gis*::init_from_json start from their actual object and don't
need a json_read_value to start.
Test case an bug report thanks to byteoverride.
The byteorder field could contain an invalid value (> 1) on a GIS
multi_point object where there byteorder was on any of the inner points.
This can occur with any of the GIS "*FromWKB" functions that contained
a multipoint object.
This result in a invalid object being accepted and also potentially
trigger undefined behaviour in the processing of the object.
Geometry::as_wkt() didn't take into account bytes written into the
result string in the recusrive call of get_data_as_wkt().
Gis_geometry_collection::get_data_as_wkt() didn't reserve space at all.
Thus nested geometry collections could overflow the buffer.
before `data+= WKB_HEADER_SIZE` verify that there's enough data.
rewrite this line in Gis_multi_line_string::get_mbr to make clear
that it does not need this check.
st_collect read past the end of its input because it assumed
that the input was a valid geometry. Check that the length of the
input is at least long enough to possibly contain a valid geometry.
Parsing a wkb of a multi_line_string needs to validate the
wkbByteOrder field for <= wkb_ndr to be valid, otherwise
return null.
Note that wkb[0] is a char, so can be signed, hence the cast and
> 0x80 values in the test case.
When creating Geometry objects from raw bytes, validate the byte order
is not > wkb_ndr, a constant for little endian being the upper bound of
the wkbByteOrder.
The 3 validation locations are the callers to the other init_from_wkb
functions for other geometry types, which prevent an invalid value from
reaching wkb_get_uint. As wkb[0], a char can signed, use uchar for
comparision. The other value in the wkb_ndr enum is wkb_xdr being 0
which makes > wk_ndr a valid test of the two allowed values.
ST_SIMPLIFY of a multilinestring, polygon, multipolygon, or geometry
collection reserved space for the result header but omitted the four
byte element count that it then appends. This resulted in a buffer
overrun.
Reserve the full header size, including the count, in each of the four
collection simplify functions, the same fix applied for MDEV-35062 and
MDEV-36042.
Geometry::create_from_json reset the killedptr and the stack
depth (stack_p) every time json_scan_start was called. We save/
restore these values so that the ST_GeomFrom_GeoJSON can be killed
and incur timeout, and also so that its depth measurement is kept.
Test case an bug report thanks to byteoverride.
From the UBSAN error:
sql/spatial.cc:3364:10: runtime error: applying non-zero offset 1 to null pointer
In Gis_geometry_collection::init_from_opresult, a pointer argument
was being treated as a counter for the Special case of
GEOMETRYCOLLECTION EMPTY. The memory location was never accessed.
Rather than use points to count and return a difference at the end, the
code is replace to use g_len_total as a counter. This gets a 1 value for
the GEOMETRYCOLLECTION EMPTY case and no ointer undefined behaviours
occur.
As other init_from_opresult functions return uint both g_len and
result use that type.
ST_AsGeoJSON stripped the first '[' bracket when serializing an EMPTY
GeometryCollection, outputting '{"type": "GeometryCollection",
"geometries":]}'.
This occurred because several Gis_*::get_data_as_json functions
unconditionally truncated the last 2 characters of their String buffer
to remove the trailing ', ' appended inside their element loops. For
empty collections, the loop doesn't execute and there is no trailing
comma, causing the truncate to strip the initial '[' bracket.
Fixed by checking if the last character in the buffer is ' ' before
truncating.
The GIS function ST_Validate takes ad input a geometry and verifies that
- is compliant with the Well-Known Binary (WKB) format and Spatial
Reference System Identifier (SRID) syntax.
- is geometrically valid.
If the input is valid return it, else it returns NULL. The use case of
this function is to filter out invalid geometry data.
Author: StefanoPetrilli <stefanop_1999@hotmail.it>
Co-authored-by: Torje Digernes <torje.digernes@oracle.com>
Co-authored-by: Hans H Melby <hans.h.melby@oracle.com>
Co-authored-by: Jon Olav Hauglid <jon.hauglid@oracle.com>
Co-authored-by: Erlend Dahl <erlend.dahl@oracle.com>
Co-authored-by: Norvald H. Ryeng <norvald.ryeng@oracle.com>
Co-authored-by: David.Zhao <david.zhao@oracle.com>
Co-authored-by: Pavan <pavan.naik@oracle.com>
The GIS function ST_Simplify takes ad input a geometry and a double. It
applies the Ramer-Douglas-Peucker algorithm on the geometry and returns
the resulting geometry. The tests have been cherry-picked from the MySQL
implementation of this function to grant compatibility among the two
implementations.
Co-authored-by: David Zhao <david.zhao@oracle.com>
Co-authored-by: Pavan Naik <pavan.naik@oracle.com>
Co-authored-by: Norvald H. Ryeng <norvald.ryeng@oracle.com>
Co-authored-by: Erlend Dahl <erlend.dahl@oracle.com>
Co-authored-by: Jon Hauglid <jon.hauglid@oracle.com>
Co-authored-by: Hans H Melby <hans.h.melby@oracle.com>
Co-authored-by: Tor Didriksen <tor.didriksen@oracle.com>
