0s autopkgtest [16:07:09]: starting date and time: 2024-07-26 16:07:09+0000 0s autopkgtest [16:07:09]: git checkout: fd3bed09 nova: allow more retries for quota issues 0s autopkgtest [16:07:09]: host juju-7f2275-prod-proposed-migration-environment-3; command line: /home/ubuntu/autopkgtest/runner/autopkgtest --output-dir /tmp/autopkgtest-work.b6hjpn4j/out --timeout-copy=6000 --setup-commands /home/ubuntu/autopkgtest-cloud/worker-config-production/setup-canonical.sh --apt-pocket=proposed=src:geoalchemy2 --apt-upgrade geoalchemy2 --timeout-short=300 --timeout-copy=20000 --timeout-build=20000 --env=ADT_TEST_TRIGGERS=geoalchemy2/0.15.1-1 -- ssh -s /home/ubuntu/autopkgtest/ssh-setup/nova -- --flavor autopkgtest --security-groups autopkgtest-juju-7f2275-prod-proposed-migration-environment-3@bos01-s390x-4.secgroup --name adt-oracular-s390x-geoalchemy2-20240726-160706-juju-7f2275-prod-proposed-migration-environment-3-87a53fad-b164-42ce-85a9-f063c5a8caeb --image adt/ubuntu-oracular-s390x-server --keyname testbed-juju-7f2275-prod-proposed-migration-environment-3 --net-id=net_prod-proposed-migration -e TERM=linux -e ''"'"'http_proxy=http://squid.internal:3128'"'"'' -e ''"'"'https_proxy=http://squid.internal:3128'"'"'' -e ''"'"'no_proxy=127.0.0.1,127.0.1.1,login.ubuntu.com,localhost,localdomain,novalocal,internal,archive.ubuntu.com,ports.ubuntu.com,security.ubuntu.com,ddebs.ubuntu.com,changelogs.ubuntu.com,keyserver.ubuntu.com,launchpadlibrarian.net,launchpadcontent.net,launchpad.net,10.24.0.0/24,keystone.ps5.canonical.com,objectstorage.prodstack5.canonical.com'"'"'' --mirror=http://us.ports.ubuntu.com/ubuntu-ports/ 133s autopkgtest [16:09:22]: testbed dpkg architecture: s390x 133s autopkgtest [16:09:22]: testbed apt version: 2.9.6 133s autopkgtest [16:09:22]: @@@@@@@@@@@@@@@@@@@@ test bed setup 134s Get:1 http://ftpmaster.internal/ubuntu oracular-proposed InRelease [126 kB] 134s Get:2 http://ftpmaster.internal/ubuntu oracular-proposed/multiverse Sources [4288 B] 134s Get:3 http://ftpmaster.internal/ubuntu oracular-proposed/main Sources [50.6 kB] 134s Get:4 http://ftpmaster.internal/ubuntu oracular-proposed/universe Sources [436 kB] 134s Get:5 http://ftpmaster.internal/ubuntu oracular-proposed/restricted Sources [8548 B] 134s Get:6 http://ftpmaster.internal/ubuntu oracular-proposed/main s390x Packages [66.8 kB] 134s Get:7 http://ftpmaster.internal/ubuntu oracular-proposed/main s390x c-n-f Metadata [2112 B] 134s Get:8 http://ftpmaster.internal/ubuntu oracular-proposed/restricted s390x Packages [1368 B] 134s Get:9 http://ftpmaster.internal/ubuntu oracular-proposed/restricted s390x c-n-f Metadata [120 B] 134s Get:10 http://ftpmaster.internal/ubuntu oracular-proposed/universe s390x Packages [397 kB] 134s Get:11 http://ftpmaster.internal/ubuntu oracular-proposed/universe s390x c-n-f Metadata [8372 B] 134s Get:12 http://ftpmaster.internal/ubuntu oracular-proposed/multiverse s390x Packages [2072 B] 134s Get:13 http://ftpmaster.internal/ubuntu oracular-proposed/multiverse s390x c-n-f Metadata [120 B] 135s Fetched 1104 kB in 1s (1075 kB/s) 135s Reading package lists... 137s Reading package lists... 138s Building dependency tree... 138s Reading state information... 138s Calculating upgrade... 138s The following packages will be upgraded: 138s bpfcc-tools bpftrace libbpfcc liblz4-1 mawk python3-bpfcc 138s 6 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 138s Need to get 17.4 MB of archives. 138s After this operation, 58.9 MB of additional disk space will be used. 138s Get:1 http://ftpmaster.internal/ubuntu oracular/main s390x liblz4-1 s390x 1.9.4-3 [75.5 kB] 138s Get:2 http://ftpmaster.internal/ubuntu oracular/main s390x mawk s390x 1.3.4.20240622-2 [132 kB] 138s Get:3 http://ftpmaster.internal/ubuntu oracular/main s390x libbpfcc s390x 0.30.0+ds-1ubuntu2 [677 kB] 139s Get:4 http://ftpmaster.internal/ubuntu oracular/main s390x python3-bpfcc all 0.30.0+ds-1ubuntu2 [40.3 kB] 139s Get:5 http://ftpmaster.internal/ubuntu oracular/main s390x bpfcc-tools all 0.30.0+ds-1ubuntu2 [697 kB] 139s Get:6 http://ftpmaster.internal/ubuntu oracular/main s390x bpftrace s390x 0.21.0-1ubuntu6 [15.8 MB] 141s Fetched 17.4 MB in 3s (5958 kB/s) 141s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 54767 files and directories currently installed.) 141s Preparing to unpack .../liblz4-1_1.9.4-3_s390x.deb ... 141s Unpacking liblz4-1:s390x (1.9.4-3) over (1.9.4-2) ... 142s Setting up liblz4-1:s390x (1.9.4-3) ... 142s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 54767 files and directories currently installed.) 142s Preparing to unpack .../mawk_1.3.4.20240622-2_s390x.deb ... 142s Unpacking mawk (1.3.4.20240622-2) over (1.3.4.20240622-1) ... 142s Preparing to unpack .../libbpfcc_0.30.0+ds-1ubuntu2_s390x.deb ... 142s Unpacking libbpfcc:s390x (0.30.0+ds-1ubuntu2) over (0.29.1+ds-1ubuntu7) ... 142s Preparing to unpack .../python3-bpfcc_0.30.0+ds-1ubuntu2_all.deb ... 142s Unpacking python3-bpfcc (0.30.0+ds-1ubuntu2) over (0.29.1+ds-1ubuntu7) ... 142s Preparing to unpack .../bpfcc-tools_0.30.0+ds-1ubuntu2_all.deb ... 142s Unpacking bpfcc-tools (0.30.0+ds-1ubuntu2) over (0.29.1+ds-1ubuntu7) ... 142s Preparing to unpack .../bpftrace_0.21.0-1ubuntu6_s390x.deb ... 142s Unpacking bpftrace (0.21.0-1ubuntu6) over (0.20.2-1ubuntu4) ... 142s Setting up mawk (1.3.4.20240622-2) ... 142s Setting up libbpfcc:s390x (0.30.0+ds-1ubuntu2) ... 142s Setting up python3-bpfcc (0.30.0+ds-1ubuntu2) ... 143s Setting up bpftrace (0.21.0-1ubuntu6) ... 143s Setting up bpfcc-tools (0.30.0+ds-1ubuntu2) ... 143s Processing triggers for man-db (2.12.1-2) ... 143s Processing triggers for libc-bin (2.39-0ubuntu9) ... 144s Reading package lists... 144s Building dependency tree... 144s Reading state information... 144s 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 145s Hit:1 http://ftpmaster.internal/ubuntu oracular-proposed InRelease 145s Hit:2 http://ftpmaster.internal/ubuntu oracular InRelease 145s Hit:3 http://ftpmaster.internal/ubuntu oracular-updates InRelease 145s Hit:4 http://ftpmaster.internal/ubuntu oracular-security InRelease 146s Reading package lists... 146s Reading package lists... 146s Building dependency tree... 146s Reading state information... 146s Calculating upgrade... 146s 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 146s Reading package lists... 147s Building dependency tree... 147s Reading state information... 147s 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 150s autopkgtest [16:09:39]: testbed running kernel: Linux 6.8.0-31-generic #31-Ubuntu SMP Sat Apr 20 00:14:26 UTC 2024 150s autopkgtest [16:09:39]: @@@@@@@@@@@@@@@@@@@@ apt-source geoalchemy2 152s Get:1 http://ftpmaster.internal/ubuntu oracular-proposed/universe geoalchemy2 0.15.1-1 (dsc) [2597 B] 152s Get:2 http://ftpmaster.internal/ubuntu oracular-proposed/universe geoalchemy2 0.15.1-1 (tar) [219 kB] 152s Get:3 http://ftpmaster.internal/ubuntu oracular-proposed/universe geoalchemy2 0.15.1-1 (diff) [8328 B] 152s gpgv: Signature made Sun May 19 11:34:07 2024 UTC 152s gpgv: using RSA key FB8ACFA78C726089C38AD0269605A1098C63B92A 152s gpgv: Can't check signature: No public key 152s dpkg-source: warning: cannot verify inline signature for ./geoalchemy2_0.15.1-1.dsc: no acceptable signature found 152s autopkgtest [16:09:41]: testing package geoalchemy2 version 0.15.1-1 153s autopkgtest [16:09:42]: build not needed 153s autopkgtest [16:09:42]: test setup-db-and-run-tests.py: preparing testbed 154s Reading package lists... 154s Building dependency tree... 154s Reading state information... 155s Starting pkgProblemResolver with broken count: 0 155s Starting 2 pkgProblemResolver with broken count: 0 155s Done 155s The following additional packages will be installed: 155s gdal-data gdal-plugins libaec0 libarmadillo14 libarpack2t64 libblas3 155s libblosc1 libcfitsio10t64 libfreexl1 libfyba0t64 libgdal35 libgeos-c1t64 155s libgeos3.12.2 libgeotiff5 libgfortran5 libgif7 libhdf4-0-alt 155s libhdf5-103-1t64 libhdf5-hl-100t64 libjs-jquery libjs-sphinxdoc 155s libjs-underscore libjson-perl libkmlbase1t64 libkmldom1t64 libkmlengine1t64 155s liblapack3 liblcms2-2 libltdl7 libminizip1t64 libmysqlclient21 155s libnetcdf19t64 libodbc2 libodbcinst2 libogdi4.1 libopenjp2-7 libpoppler137 155s libpq5 libproj25 libqhull-r8.0 librttopo1 libsnappy1v5 libspatialite8t64 155s libsuperlu6 libsz2 liburiparser1 libxerces-c3.2t64 libxslt1.1 mysql-common 155s postgis postgresql-16 postgresql-16-postgis-3 155s postgresql-16-postgis-3-scripts postgresql-client-16 155s postgresql-client-common postgresql-common postgresql-postgis 155s postgresql-postgis-scripts proj-data python-geoalchemy2-doc python3-affine 155s python3-alembic python3-click python3-click-plugins python3-cligj 155s python3-colorama python3-geoalchemy2 python3-greenlet python3-iniconfig 155s python3-mako python3-mysqldb python3-numpy python3-packaging python3-pluggy 155s python3-psycopg2 python3-pytest python3-rasterio python3-shapely 155s python3-snuggs python3-sqlalchemy ssl-cert unixodbc-common 155s Suggested packages: 155s geotiff-bin gdal-bin libgeotiff-epsg libhdf4-alt-dev hdf4-tools 155s liblcms2-utils odbc-postgresql tdsodbc ogdi-bin proj-bin postgresql-doc-16 155s python-greenlet-dev python-greenlet-doc python-mako-doc python3-beaker 155s default-mysql-server | virtual-mysql-server gcc gfortran python-numpy-doc 155s python3-dev python-psycopg2-doc python3-matplotlib python-sqlalchemy-doc 155s python3-fdb python3-pymssql python3-asyncpg python3-aiosqlite 155s Recommended packages: 155s proj-bin javascript-common libjson-xs-perl poppler-data postgis-doc 155s python3-sqlalchemy-ext 155s The following NEW packages will be installed: 155s autopkgtest-satdep gdal-data gdal-plugins libaec0 libarmadillo14 155s libarpack2t64 libblas3 libblosc1 libcfitsio10t64 libfreexl1 libfyba0t64 155s libgdal35 libgeos-c1t64 libgeos3.12.2 libgeotiff5 libgfortran5 libgif7 155s libhdf4-0-alt libhdf5-103-1t64 libhdf5-hl-100t64 libjs-jquery 155s libjs-sphinxdoc libjs-underscore libjson-perl libkmlbase1t64 libkmldom1t64 155s libkmlengine1t64 liblapack3 liblcms2-2 libltdl7 libminizip1t64 155s libmysqlclient21 libnetcdf19t64 libodbc2 libodbcinst2 libogdi4.1 155s libopenjp2-7 libpoppler137 libpq5 libproj25 libqhull-r8.0 librttopo1 155s libsnappy1v5 libspatialite8t64 libsuperlu6 libsz2 liburiparser1 155s libxerces-c3.2t64 libxslt1.1 mysql-common postgis postgresql-16 155s postgresql-16-postgis-3 postgresql-16-postgis-3-scripts postgresql-client-16 155s postgresql-client-common postgresql-common postgresql-postgis 155s postgresql-postgis-scripts proj-data python-geoalchemy2-doc python3-affine 155s python3-alembic python3-click python3-click-plugins python3-cligj 155s python3-colorama python3-geoalchemy2 python3-greenlet python3-iniconfig 155s python3-mako python3-mysqldb python3-numpy python3-packaging python3-pluggy 155s python3-psycopg2 python3-pytest python3-rasterio python3-shapely 155s python3-snuggs python3-sqlalchemy ssl-cert unixodbc-common 155s 0 upgraded, 83 newly installed, 0 to remove and 0 not upgraded. 155s Need to get 69.4 MB/69.4 MB of archives. 155s After this operation, 267 MB of additional disk space will be used. 155s Get:1 /tmp/autopkgtest.fTasHg/1-autopkgtest-satdep.deb autopkgtest-satdep s390x 0 [788 B] 155s Get:2 http://ftpmaster.internal/ubuntu oracular/main s390x libjson-perl all 4.10000-1 [81.9 kB] 156s Get:3 http://ftpmaster.internal/ubuntu oracular/main s390x postgresql-client-common all 261 [36.6 kB] 156s Get:4 http://ftpmaster.internal/ubuntu oracular/main s390x ssl-cert all 1.1.2ubuntu2 [18.0 kB] 156s Get:5 http://ftpmaster.internal/ubuntu oracular/main s390x postgresql-common all 261 [162 kB] 156s Get:6 http://ftpmaster.internal/ubuntu oracular/universe s390x gdal-data all 3.9.1+dfsg-1build1 [269 kB] 156s Get:7 http://ftpmaster.internal/ubuntu oracular/universe s390x gdal-plugins s390x 3.9.1+dfsg-1build1 [25.2 kB] 156s Get:8 http://ftpmaster.internal/ubuntu oracular/universe s390x libaec0 s390x 1.1.3-1 [25.7 kB] 156s Get:9 http://ftpmaster.internal/ubuntu oracular/main s390x libblas3 s390x 3.12.0-3build1 [245 kB] 156s Get:10 http://ftpmaster.internal/ubuntu oracular/main s390x libgfortran5 s390x 14.1.0-5ubuntu1 [587 kB] 156s Get:11 http://ftpmaster.internal/ubuntu oracular/main s390x liblapack3 s390x 3.12.0-3build1 [2976 kB] 156s Get:12 http://ftpmaster.internal/ubuntu oracular/universe s390x libarpack2t64 s390x 3.9.1-1.1build2 [113 kB] 156s Get:13 http://ftpmaster.internal/ubuntu oracular/universe s390x libsuperlu6 s390x 6.0.1+dfsg1-1build1 [232 kB] 156s Get:14 http://ftpmaster.internal/ubuntu oracular/universe s390x libarmadillo14 s390x 1:14.0.1+dfsg-2 [110 kB] 156s Get:15 http://ftpmaster.internal/ubuntu oracular/main s390x libsnappy1v5 s390x 1.2.1-1 [33.0 kB] 156s Get:16 http://ftpmaster.internal/ubuntu oracular/universe s390x libblosc1 s390x 1.21.5+ds-1build1 [27.7 kB] 156s Get:17 http://ftpmaster.internal/ubuntu oracular/universe s390x libcfitsio10t64 s390x 4.4.1-2 [595 kB] 156s Get:18 http://ftpmaster.internal/ubuntu oracular/universe s390x libfyba0t64 s390x 4.1.1-11build1 [125 kB] 156s Get:19 http://ftpmaster.internal/ubuntu oracular/universe s390x libminizip1t64 s390x 1:1.3.dfsg-3.1ubuntu2 [24.4 kB] 156s Get:20 http://ftpmaster.internal/ubuntu oracular/universe s390x libfreexl1 s390x 2.0.0-1build2 [46.5 kB] 156s Get:21 http://ftpmaster.internal/ubuntu oracular/universe s390x libgeos3.12.2 s390x 3.12.2-1 [883 kB] 156s Get:22 http://ftpmaster.internal/ubuntu oracular/universe s390x libgeos-c1t64 s390x 3.12.2-1 [90.8 kB] 156s Get:23 http://ftpmaster.internal/ubuntu oracular/universe s390x proj-data all 9.4.1-1 [7892 kB] 157s Get:24 http://ftpmaster.internal/ubuntu oracular/universe s390x libproj25 s390x 9.4.1-1 [1398 kB] 157s Get:25 http://ftpmaster.internal/ubuntu oracular/universe s390x libgeotiff5 s390x 1.7.3-1 [66.4 kB] 157s Get:26 http://ftpmaster.internal/ubuntu oracular/main s390x libgif7 s390x 5.2.2-1ubuntu1 [38.0 kB] 157s Get:27 http://ftpmaster.internal/ubuntu oracular/universe s390x libsz2 s390x 1.1.3-1 [5442 B] 157s Get:28 http://ftpmaster.internal/ubuntu oracular/universe s390x libhdf4-0-alt s390x 4.3.0-1 [300 kB] 157s Get:29 http://ftpmaster.internal/ubuntu oracular/universe s390x libhdf5-103-1t64 s390x 1.10.10+repack-3.1ubuntu4 [1436 kB] 157s Get:30 http://ftpmaster.internal/ubuntu oracular/universe s390x liburiparser1 s390x 0.9.8+dfsg-1 [41.0 kB] 157s Get:31 http://ftpmaster.internal/ubuntu oracular/universe s390x libkmlbase1t64 s390x 1.3.0-12build1 [50.5 kB] 157s Get:32 http://ftpmaster.internal/ubuntu oracular/universe s390x libkmldom1t64 s390x 1.3.0-12build1 [151 kB] 157s Get:33 http://ftpmaster.internal/ubuntu oracular/universe s390x libkmlengine1t64 s390x 1.3.0-12build1 [68.9 kB] 157s Get:34 http://ftpmaster.internal/ubuntu oracular/main s390x mysql-common all 5.8+1.1.1 [6800 B] 157s Get:35 http://ftpmaster.internal/ubuntu oracular/main s390x libmysqlclient21 s390x 8.0.38-1 [1295 kB] 157s Get:36 http://ftpmaster.internal/ubuntu oracular/universe s390x libhdf5-hl-100t64 s390x 1.10.10+repack-3.1ubuntu4 [58.0 kB] 157s Get:37 http://ftpmaster.internal/ubuntu oracular/universe s390x libnetcdf19t64 s390x 1:4.9.2-6 [498 kB] 157s Get:38 http://ftpmaster.internal/ubuntu oracular/main s390x libltdl7 s390x 2.4.7-7build1 [41.8 kB] 157s Get:39 http://ftpmaster.internal/ubuntu oracular/main s390x libodbc2 s390x 2.3.12-1ubuntu1 [162 kB] 157s Get:40 http://ftpmaster.internal/ubuntu oracular/main s390x unixodbc-common all 2.3.12-1ubuntu1 [8796 B] 157s Get:41 http://ftpmaster.internal/ubuntu oracular/main s390x libodbcinst2 s390x 2.3.12-1ubuntu1 [32.0 kB] 157s Get:42 http://ftpmaster.internal/ubuntu oracular/universe s390x libogdi4.1 s390x 4.1.1+ds-4 [220 kB] 157s Get:43 http://ftpmaster.internal/ubuntu oracular/main s390x libopenjp2-7 s390x 2.5.0-2build3 [192 kB] 157s Get:44 http://ftpmaster.internal/ubuntu oracular/main s390x liblcms2-2 s390x 2.14-2build1 [172 kB] 157s Get:45 http://ftpmaster.internal/ubuntu oracular/main s390x libpoppler137 s390x 24.06.0-2 [1176 kB] 157s Get:46 http://ftpmaster.internal/ubuntu oracular/main s390x libpq5 s390x 16.3-1 [144 kB] 157s Get:47 http://ftpmaster.internal/ubuntu oracular/universe s390x libqhull-r8.0 s390x 2020.2-6build1 [199 kB] 157s Get:48 http://ftpmaster.internal/ubuntu oracular/universe s390x librttopo1 s390x 1.1.0-3build2 [200 kB] 157s Get:49 http://ftpmaster.internal/ubuntu oracular/universe s390x libspatialite8t64 s390x 5.1.0-3build1 [2050 kB] 158s Get:50 http://ftpmaster.internal/ubuntu oracular/universe s390x libxerces-c3.2t64 s390x 3.2.4+debian-1.3 [942 kB] 158s Get:51 http://ftpmaster.internal/ubuntu oracular/universe s390x libgdal35 s390x 3.9.1+dfsg-1build1 [9078 kB] 158s Get:52 http://ftpmaster.internal/ubuntu oracular/main s390x libjs-jquery all 3.6.1+dfsg+~3.5.14-1 [328 kB] 158s Get:53 http://ftpmaster.internal/ubuntu oracular/main s390x libjs-underscore all 1.13.4~dfsg+~1.11.4-3 [118 kB] 158s Get:54 http://ftpmaster.internal/ubuntu oracular/main s390x libjs-sphinxdoc all 7.3.7-3 [154 kB] 158s Get:55 http://ftpmaster.internal/ubuntu oracular/main s390x libxslt1.1 s390x 1.1.39-0exp1build1 [170 kB] 158s Get:56 http://ftpmaster.internal/ubuntu oracular/universe s390x postgis s390x 3.4.2+dfsg-2 [168 kB] 158s Get:57 http://ftpmaster.internal/ubuntu oracular/main s390x postgresql-client-16 s390x 16.3-1 [1290 kB] 158s Get:58 http://ftpmaster.internal/ubuntu oracular/main s390x postgresql-16 s390x 16.3-1 [16.7 MB] 159s Get:59 http://ftpmaster.internal/ubuntu oracular/universe s390x postgresql-16-postgis-3-scripts all 3.4.2+dfsg-2 [992 kB] 159s Get:60 http://ftpmaster.internal/ubuntu oracular/universe s390x postgresql-16-postgis-3 s390x 3.4.2+dfsg-2 [7016 kB] 159s Get:61 http://ftpmaster.internal/ubuntu oracular/universe s390x postgresql-postgis-scripts all 3.4.2+dfsg-2 [14.4 kB] 159s Get:62 http://ftpmaster.internal/ubuntu oracular/universe s390x postgresql-postgis s390x 3.4.2+dfsg-2 [14.4 kB] 159s Get:63 http://ftpmaster.internal/ubuntu oracular-proposed/universe s390x python-geoalchemy2-doc all 0.15.1-1 [272 kB] 159s Get:64 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-affine all 2.4.0-1 [14.3 kB] 159s Get:65 http://ftpmaster.internal/ubuntu oracular/main s390x python3-mako all 1.3.5-1 [63.4 kB] 159s Get:66 http://ftpmaster.internal/ubuntu oracular/main s390x python3-greenlet s390x 3.0.3-0ubuntu5 [156 kB] 159s Get:67 http://ftpmaster.internal/ubuntu oracular/main s390x python3-sqlalchemy all 1.4.50+ds1-1build1 [1020 kB] 159s Get:68 http://ftpmaster.internal/ubuntu oracular/main s390x python3-alembic all 1.13.1-4 [149 kB] 159s Get:69 http://ftpmaster.internal/ubuntu oracular/main s390x python3-colorama all 0.4.6-4 [32.1 kB] 159s Get:70 http://ftpmaster.internal/ubuntu oracular/main s390x python3-click all 8.1.7-1 [79.1 kB] 159s Get:71 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-click-plugins all 1.1.1-4 [9288 B] 159s Get:72 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-cligj all 0.7.2-2 [7788 B] 159s Get:73 http://ftpmaster.internal/ubuntu oracular/main s390x python3-packaging all 24.1-1 [41.4 kB] 159s Get:74 http://ftpmaster.internal/ubuntu oracular-proposed/universe s390x python3-geoalchemy2 all 0.15.1-1 [51.1 kB] 159s Get:75 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-iniconfig all 1.1.1-2 [6024 B] 159s Get:76 http://ftpmaster.internal/ubuntu oracular/main s390x python3-mysqldb s390x 1.4.6-2build3 [47.5 kB] 159s Get:77 http://ftpmaster.internal/ubuntu oracular/main s390x python3-numpy s390x 1:1.26.4+ds-10 [4090 kB] 159s Get:78 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-pluggy all 1.5.0-1 [21.0 kB] 159s Get:79 http://ftpmaster.internal/ubuntu oracular/main s390x python3-psycopg2 s390x 2.9.9-1build1 [133 kB] 159s Get:80 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-pytest all 7.4.4-1 [305 kB] 159s Get:81 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-snuggs all 1.4.7-4 [6456 B] 159s Get:82 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-rasterio s390x 1.3.10-2build1 [1005 kB] 160s Get:83 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-shapely s390x 2.0.5-2 [309 kB] 160s Preconfiguring packages ... 160s Fetched 69.4 MB in 5s (15.3 MB/s) 160s Selecting previously unselected package libjson-perl. 160s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 54774 files and directories currently installed.) 160s Preparing to unpack .../00-libjson-perl_4.10000-1_all.deb ... 160s Unpacking libjson-perl (4.10000-1) ... 160s Selecting previously unselected package postgresql-client-common. 160s Preparing to unpack .../01-postgresql-client-common_261_all.deb ... 160s Unpacking postgresql-client-common (261) ... 160s Selecting previously unselected package ssl-cert. 160s Preparing to unpack .../02-ssl-cert_1.1.2ubuntu2_all.deb ... 160s Unpacking ssl-cert (1.1.2ubuntu2) ... 160s Selecting previously unselected package postgresql-common. 160s Preparing to unpack .../03-postgresql-common_261_all.deb ... 160s Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 160s Unpacking postgresql-common (261) ... 160s Selecting previously unselected package gdal-data. 160s Preparing to unpack .../04-gdal-data_3.9.1+dfsg-1build1_all.deb ... 160s Unpacking gdal-data (3.9.1+dfsg-1build1) ... 160s Selecting previously unselected package gdal-plugins:s390x. 160s Preparing to unpack .../05-gdal-plugins_3.9.1+dfsg-1build1_s390x.deb ... 160s Unpacking gdal-plugins:s390x (3.9.1+dfsg-1build1) ... 160s Selecting previously unselected package libaec0:s390x. 160s Preparing to unpack .../06-libaec0_1.1.3-1_s390x.deb ... 160s Unpacking libaec0:s390x (1.1.3-1) ... 160s Selecting previously unselected package libblas3:s390x. 160s Preparing to unpack .../07-libblas3_3.12.0-3build1_s390x.deb ... 160s Unpacking libblas3:s390x (3.12.0-3build1) ... 160s Selecting previously unselected package libgfortran5:s390x. 160s Preparing to unpack .../08-libgfortran5_14.1.0-5ubuntu1_s390x.deb ... 160s Unpacking libgfortran5:s390x (14.1.0-5ubuntu1) ... 160s Selecting previously unselected package liblapack3:s390x. 160s Preparing to unpack .../09-liblapack3_3.12.0-3build1_s390x.deb ... 160s Unpacking liblapack3:s390x (3.12.0-3build1) ... 161s Selecting previously unselected package libarpack2t64:s390x. 161s Preparing to unpack .../10-libarpack2t64_3.9.1-1.1build2_s390x.deb ... 161s Unpacking libarpack2t64:s390x (3.9.1-1.1build2) ... 161s Selecting previously unselected package libsuperlu6:s390x. 161s Preparing to unpack .../11-libsuperlu6_6.0.1+dfsg1-1build1_s390x.deb ... 161s Unpacking libsuperlu6:s390x (6.0.1+dfsg1-1build1) ... 161s Selecting previously unselected package libarmadillo14. 161s Preparing to unpack .../12-libarmadillo14_1%3a14.0.1+dfsg-2_s390x.deb ... 161s Unpacking libarmadillo14 (1:14.0.1+dfsg-2) ... 161s Selecting previously unselected package libsnappy1v5:s390x. 161s Preparing to unpack .../13-libsnappy1v5_1.2.1-1_s390x.deb ... 161s Unpacking libsnappy1v5:s390x (1.2.1-1) ... 161s Selecting previously unselected package libblosc1:s390x. 161s Preparing to unpack .../14-libblosc1_1.21.5+ds-1build1_s390x.deb ... 161s Unpacking libblosc1:s390x (1.21.5+ds-1build1) ... 161s Selecting previously unselected package libcfitsio10t64:s390x. 161s Preparing to unpack .../15-libcfitsio10t64_4.4.1-2_s390x.deb ... 161s Unpacking libcfitsio10t64:s390x (4.4.1-2) ... 161s Selecting previously unselected package libfyba0t64:s390x. 161s Preparing to unpack .../16-libfyba0t64_4.1.1-11build1_s390x.deb ... 161s Unpacking libfyba0t64:s390x (4.1.1-11build1) ... 161s Selecting previously unselected package libminizip1t64:s390x. 161s Preparing to unpack .../17-libminizip1t64_1%3a1.3.dfsg-3.1ubuntu2_s390x.deb ... 161s Unpacking libminizip1t64:s390x (1:1.3.dfsg-3.1ubuntu2) ... 161s Selecting previously unselected package libfreexl1:s390x. 161s Preparing to unpack .../18-libfreexl1_2.0.0-1build2_s390x.deb ... 161s Unpacking libfreexl1:s390x (2.0.0-1build2) ... 161s Selecting previously unselected package libgeos3.12.2:s390x. 161s Preparing to unpack .../19-libgeos3.12.2_3.12.2-1_s390x.deb ... 161s Unpacking libgeos3.12.2:s390x (3.12.2-1) ... 161s Selecting previously unselected package libgeos-c1t64:s390x. 161s Preparing to unpack .../20-libgeos-c1t64_3.12.2-1_s390x.deb ... 161s Unpacking libgeos-c1t64:s390x (3.12.2-1) ... 161s Selecting previously unselected package proj-data. 161s Preparing to unpack .../21-proj-data_9.4.1-1_all.deb ... 161s Unpacking proj-data (9.4.1-1) ... 161s Selecting previously unselected package libproj25:s390x. 161s Preparing to unpack .../22-libproj25_9.4.1-1_s390x.deb ... 161s Unpacking libproj25:s390x (9.4.1-1) ... 161s Selecting previously unselected package libgeotiff5:s390x. 161s Preparing to unpack .../23-libgeotiff5_1.7.3-1_s390x.deb ... 161s Unpacking libgeotiff5:s390x (1.7.3-1) ... 161s Selecting previously unselected package libgif7:s390x. 161s Preparing to unpack .../24-libgif7_5.2.2-1ubuntu1_s390x.deb ... 161s Unpacking libgif7:s390x (5.2.2-1ubuntu1) ... 161s Selecting previously unselected package libsz2:s390x. 161s Preparing to unpack .../25-libsz2_1.1.3-1_s390x.deb ... 161s Unpacking libsz2:s390x (1.1.3-1) ... 161s Selecting previously unselected package libhdf4-0-alt:s390x. 161s Preparing to unpack .../26-libhdf4-0-alt_4.3.0-1_s390x.deb ... 161s Unpacking libhdf4-0-alt:s390x (4.3.0-1) ... 161s Selecting previously unselected package libhdf5-103-1t64:s390x. 161s Preparing to unpack .../27-libhdf5-103-1t64_1.10.10+repack-3.1ubuntu4_s390x.deb ... 161s Unpacking libhdf5-103-1t64:s390x (1.10.10+repack-3.1ubuntu4) ... 161s Selecting previously unselected package liburiparser1:s390x. 161s Preparing to unpack .../28-liburiparser1_0.9.8+dfsg-1_s390x.deb ... 161s Unpacking liburiparser1:s390x (0.9.8+dfsg-1) ... 161s Selecting previously unselected package libkmlbase1t64:s390x. 161s Preparing to unpack .../29-libkmlbase1t64_1.3.0-12build1_s390x.deb ... 161s Unpacking libkmlbase1t64:s390x (1.3.0-12build1) ... 161s Selecting previously unselected package libkmldom1t64:s390x. 161s Preparing to unpack .../30-libkmldom1t64_1.3.0-12build1_s390x.deb ... 161s Unpacking libkmldom1t64:s390x (1.3.0-12build1) ... 161s Selecting previously unselected package libkmlengine1t64:s390x. 161s Preparing to unpack .../31-libkmlengine1t64_1.3.0-12build1_s390x.deb ... 161s Unpacking libkmlengine1t64:s390x (1.3.0-12build1) ... 161s Selecting previously unselected package mysql-common. 161s Preparing to unpack .../32-mysql-common_5.8+1.1.1_all.deb ... 161s Unpacking mysql-common (5.8+1.1.1) ... 161s Selecting previously unselected package libmysqlclient21:s390x. 161s Preparing to unpack .../33-libmysqlclient21_8.0.38-1_s390x.deb ... 161s Unpacking libmysqlclient21:s390x (8.0.38-1) ... 161s Selecting previously unselected package libhdf5-hl-100t64:s390x. 161s Preparing to unpack .../34-libhdf5-hl-100t64_1.10.10+repack-3.1ubuntu4_s390x.deb ... 161s Unpacking libhdf5-hl-100t64:s390x (1.10.10+repack-3.1ubuntu4) ... 161s Selecting previously unselected package libnetcdf19t64:s390x. 161s Preparing to unpack .../35-libnetcdf19t64_1%3a4.9.2-6_s390x.deb ... 161s Unpacking libnetcdf19t64:s390x (1:4.9.2-6) ... 161s Selecting previously unselected package libltdl7:s390x. 161s Preparing to unpack .../36-libltdl7_2.4.7-7build1_s390x.deb ... 161s Unpacking libltdl7:s390x (2.4.7-7build1) ... 161s Selecting previously unselected package libodbc2:s390x. 161s Preparing to unpack .../37-libodbc2_2.3.12-1ubuntu1_s390x.deb ... 161s Unpacking libodbc2:s390x (2.3.12-1ubuntu1) ... 161s Selecting previously unselected package unixodbc-common. 161s Preparing to unpack .../38-unixodbc-common_2.3.12-1ubuntu1_all.deb ... 161s Unpacking unixodbc-common (2.3.12-1ubuntu1) ... 162s Selecting previously unselected package libodbcinst2:s390x. 162s Preparing to unpack .../39-libodbcinst2_2.3.12-1ubuntu1_s390x.deb ... 162s Unpacking libodbcinst2:s390x (2.3.12-1ubuntu1) ... 162s Selecting previously unselected package libogdi4.1:s390x. 162s Preparing to unpack .../40-libogdi4.1_4.1.1+ds-4_s390x.deb ... 162s Unpacking libogdi4.1:s390x (4.1.1+ds-4) ... 162s Selecting previously unselected package libopenjp2-7:s390x. 162s Preparing to unpack .../41-libopenjp2-7_2.5.0-2build3_s390x.deb ... 162s Unpacking libopenjp2-7:s390x (2.5.0-2build3) ... 162s Selecting previously unselected package liblcms2-2:s390x. 162s Preparing to unpack .../42-liblcms2-2_2.14-2build1_s390x.deb ... 162s Unpacking liblcms2-2:s390x (2.14-2build1) ... 162s Selecting previously unselected package libpoppler137:s390x. 162s Preparing to unpack .../43-libpoppler137_24.06.0-2_s390x.deb ... 162s Unpacking libpoppler137:s390x (24.06.0-2) ... 162s Selecting previously unselected package libpq5:s390x. 162s Preparing to unpack .../44-libpq5_16.3-1_s390x.deb ... 162s Unpacking libpq5:s390x (16.3-1) ... 162s Selecting previously unselected package libqhull-r8.0:s390x. 162s Preparing to unpack .../45-libqhull-r8.0_2020.2-6build1_s390x.deb ... 162s Unpacking libqhull-r8.0:s390x (2020.2-6build1) ... 162s Selecting previously unselected package librttopo1:s390x. 162s Preparing to unpack .../46-librttopo1_1.1.0-3build2_s390x.deb ... 162s Unpacking librttopo1:s390x (1.1.0-3build2) ... 162s Selecting previously unselected package libspatialite8t64:s390x. 162s Preparing to unpack .../47-libspatialite8t64_5.1.0-3build1_s390x.deb ... 162s Unpacking libspatialite8t64:s390x (5.1.0-3build1) ... 162s Selecting previously unselected package libxerces-c3.2t64:s390x. 162s Preparing to unpack .../48-libxerces-c3.2t64_3.2.4+debian-1.3_s390x.deb ... 162s Unpacking libxerces-c3.2t64:s390x (3.2.4+debian-1.3) ... 162s Selecting previously unselected package libgdal35:s390x. 162s Preparing to unpack .../49-libgdal35_3.9.1+dfsg-1build1_s390x.deb ... 162s Unpacking libgdal35:s390x (3.9.1+dfsg-1build1) ... 162s Selecting previously unselected package libjs-jquery. 162s Preparing to unpack .../50-libjs-jquery_3.6.1+dfsg+~3.5.14-1_all.deb ... 162s Unpacking libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... 162s Selecting previously unselected package libjs-underscore. 162s Preparing to unpack .../51-libjs-underscore_1.13.4~dfsg+~1.11.4-3_all.deb ... 162s Unpacking libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... 162s Selecting previously unselected package libjs-sphinxdoc. 162s Preparing to unpack .../52-libjs-sphinxdoc_7.3.7-3_all.deb ... 162s Unpacking libjs-sphinxdoc (7.3.7-3) ... 162s Selecting previously unselected package libxslt1.1:s390x. 162s Preparing to unpack .../53-libxslt1.1_1.1.39-0exp1build1_s390x.deb ... 162s Unpacking libxslt1.1:s390x (1.1.39-0exp1build1) ... 162s Selecting previously unselected package postgis. 162s Preparing to unpack .../54-postgis_3.4.2+dfsg-2_s390x.deb ... 162s Unpacking postgis (3.4.2+dfsg-2) ... 162s Selecting previously unselected package postgresql-client-16. 162s Preparing to unpack .../55-postgresql-client-16_16.3-1_s390x.deb ... 162s Unpacking postgresql-client-16 (16.3-1) ... 162s Selecting previously unselected package postgresql-16. 162s Preparing to unpack .../56-postgresql-16_16.3-1_s390x.deb ... 162s Unpacking postgresql-16 (16.3-1) ... 163s Selecting previously unselected package postgresql-16-postgis-3-scripts. 163s Preparing to unpack .../57-postgresql-16-postgis-3-scripts_3.4.2+dfsg-2_all.deb ... 163s Unpacking postgresql-16-postgis-3-scripts (3.4.2+dfsg-2) ... 163s Selecting previously unselected package postgresql-16-postgis-3. 163s Preparing to unpack .../58-postgresql-16-postgis-3_3.4.2+dfsg-2_s390x.deb ... 163s Unpacking postgresql-16-postgis-3 (3.4.2+dfsg-2) ... 163s Selecting previously unselected package postgresql-postgis-scripts. 163s Preparing to unpack .../59-postgresql-postgis-scripts_3.4.2+dfsg-2_all.deb ... 163s Unpacking postgresql-postgis-scripts (3.4.2+dfsg-2) ... 163s Selecting previously unselected package postgresql-postgis. 163s Preparing to unpack .../60-postgresql-postgis_3.4.2+dfsg-2_s390x.deb ... 163s Unpacking postgresql-postgis (3.4.2+dfsg-2) ... 163s Selecting previously unselected package python-geoalchemy2-doc. 163s Preparing to unpack .../61-python-geoalchemy2-doc_0.15.1-1_all.deb ... 163s Unpacking python-geoalchemy2-doc (0.15.1-1) ... 163s Selecting previously unselected package python3-affine. 163s Preparing to unpack .../62-python3-affine_2.4.0-1_all.deb ... 163s Unpacking python3-affine (2.4.0-1) ... 163s Selecting previously unselected package python3-mako. 163s Preparing to unpack .../63-python3-mako_1.3.5-1_all.deb ... 163s Unpacking python3-mako (1.3.5-1) ... 163s Selecting previously unselected package python3-greenlet. 163s Preparing to unpack .../64-python3-greenlet_3.0.3-0ubuntu5_s390x.deb ... 163s Unpacking python3-greenlet (3.0.3-0ubuntu5) ... 163s Selecting previously unselected package python3-sqlalchemy. 163s Preparing to unpack .../65-python3-sqlalchemy_1.4.50+ds1-1build1_all.deb ... 163s Unpacking python3-sqlalchemy (1.4.50+ds1-1build1) ... 164s Selecting previously unselected package python3-alembic. 164s Preparing to unpack .../66-python3-alembic_1.13.1-4_all.deb ... 164s Unpacking python3-alembic (1.13.1-4) ... 164s Selecting previously unselected package python3-colorama. 164s Preparing to unpack .../67-python3-colorama_0.4.6-4_all.deb ... 164s Unpacking python3-colorama (0.4.6-4) ... 164s Selecting previously unselected package python3-click. 164s Preparing to unpack .../68-python3-click_8.1.7-1_all.deb ... 164s Unpacking python3-click (8.1.7-1) ... 164s Selecting previously unselected package python3-click-plugins. 164s Preparing to unpack .../69-python3-click-plugins_1.1.1-4_all.deb ... 164s Unpacking python3-click-plugins (1.1.1-4) ... 164s Selecting previously unselected package python3-cligj. 164s Preparing to unpack .../70-python3-cligj_0.7.2-2_all.deb ... 164s Unpacking python3-cligj (0.7.2-2) ... 164s Selecting previously unselected package python3-packaging. 164s Preparing to unpack .../71-python3-packaging_24.1-1_all.deb ... 164s Unpacking python3-packaging (24.1-1) ... 164s Selecting previously unselected package python3-geoalchemy2. 164s Preparing to unpack .../72-python3-geoalchemy2_0.15.1-1_all.deb ... 164s Unpacking python3-geoalchemy2 (0.15.1-1) ... 164s Selecting previously unselected package python3-iniconfig. 164s Preparing to unpack .../73-python3-iniconfig_1.1.1-2_all.deb ... 164s Unpacking python3-iniconfig (1.1.1-2) ... 164s Selecting previously unselected package python3-mysqldb. 164s Preparing to unpack .../74-python3-mysqldb_1.4.6-2build3_s390x.deb ... 164s Unpacking python3-mysqldb (1.4.6-2build3) ... 164s Selecting previously unselected package python3-numpy. 164s Preparing to unpack .../75-python3-numpy_1%3a1.26.4+ds-10_s390x.deb ... 164s Unpacking python3-numpy (1:1.26.4+ds-10) ... 164s Selecting previously unselected package python3-pluggy. 164s Preparing to unpack .../76-python3-pluggy_1.5.0-1_all.deb ... 164s Unpacking python3-pluggy (1.5.0-1) ... 164s Selecting previously unselected package python3-psycopg2. 164s Preparing to unpack .../77-python3-psycopg2_2.9.9-1build1_s390x.deb ... 164s Unpacking python3-psycopg2 (2.9.9-1build1) ... 164s Selecting previously unselected package python3-pytest. 164s Preparing to unpack .../78-python3-pytest_7.4.4-1_all.deb ... 164s Unpacking python3-pytest (7.4.4-1) ... 164s Selecting previously unselected package python3-snuggs. 164s Preparing to unpack .../79-python3-snuggs_1.4.7-4_all.deb ... 164s Unpacking python3-snuggs (1.4.7-4) ... 164s Selecting previously unselected package python3-rasterio. 164s Preparing to unpack .../80-python3-rasterio_1.3.10-2build1_s390x.deb ... 164s Unpacking python3-rasterio (1.3.10-2build1) ... 164s Selecting previously unselected package python3-shapely. 164s Preparing to unpack .../81-python3-shapely_2.0.5-2_s390x.deb ... 164s Unpacking python3-shapely (2.0.5-2) ... 164s Selecting previously unselected package autopkgtest-satdep. 164s Preparing to unpack .../82-1-autopkgtest-satdep.deb ... 164s Unpacking autopkgtest-satdep (0) ... 164s Setting up postgresql-client-common (261) ... 164s Setting up liblcms2-2:s390x (2.14-2build1) ... 164s Setting up python3-iniconfig (1.1.1-2) ... 164s Setting up mysql-common (5.8+1.1.1) ... 164s update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode 164s Setting up libmysqlclient21:s390x (8.0.38-1) ... 164s Setting up proj-data (9.4.1-1) ... 164s Setting up libproj25:s390x (9.4.1-1) ... 164s Setting up libogdi4.1:s390x (4.1.1+ds-4) ... 164s Setting up python3-colorama (0.4.6-4) ... 164s Setting up libgeos3.12.2:s390x (3.12.2-1) ... 164s Setting up libpq5:s390x (16.3-1) ... 164s Setting up libqhull-r8.0:s390x (2020.2-6build1) ... 164s Setting up libxerces-c3.2t64:s390x (3.2.4+debian-1.3) ... 164s Setting up python3-click (8.1.7-1) ... 165s Setting up libaec0:s390x (1.1.3-1) ... 165s Setting up gdal-data (3.9.1+dfsg-1build1) ... 165s Setting up libgeotiff5:s390x (1.7.3-1) ... 165s Setting up libsnappy1v5:s390x (1.2.1-1) ... 165s Setting up python3-mysqldb (1.4.6-2build3) ... 165s Setting up libblas3:s390x (3.12.0-3build1) ... 165s update-alternatives: using /usr/lib/s390x-linux-gnu/blas/libblas.so.3 to provide /usr/lib/s390x-linux-gnu/libblas.so.3 (libblas.so.3-s390x-linux-gnu) in auto mode 165s Setting up python3-packaging (24.1-1) ... 165s Setting up libcfitsio10t64:s390x (4.4.1-2) ... 165s Setting up ssl-cert (1.1.2ubuntu2) ... 166s Created symlink '/etc/systemd/system/multi-user.target.wants/ssl-cert.service' → '/usr/lib/systemd/system/ssl-cert.service'. 166s Setting up python3-greenlet (3.0.3-0ubuntu5) ... 166s Setting up python3-psycopg2 (2.9.9-1build1) ... 166s Setting up python3-click-plugins (1.1.1-4) ... 167s Setting up unixodbc-common (2.3.12-1ubuntu1) ... 167s Setting up libltdl7:s390x (2.4.7-7build1) ... 167s Setting up libgfortran5:s390x (14.1.0-5ubuntu1) ... 167s Setting up python3-pluggy (1.5.0-1) ... 167s Setting up libgif7:s390x (5.2.2-1ubuntu1) ... 167s Setting up libodbc2:s390x (2.3.12-1ubuntu1) ... 167s Setting up liburiparser1:s390x (0.9.8+dfsg-1) ... 167s Setting up libfyba0t64:s390x (4.1.1-11build1) ... 167s Setting up libminizip1t64:s390x (1:1.3.dfsg-3.1ubuntu2) ... 167s Setting up libjson-perl (4.10000-1) ... 167s Setting up libxslt1.1:s390x (1.1.39-0exp1build1) ... 167s Setting up libblosc1:s390x (1.21.5+ds-1build1) ... 167s Setting up libopenjp2-7:s390x (2.5.0-2build3) ... 167s Setting up libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... 167s Setting up python3-affine (2.4.0-1) ... 167s Setting up libkmlbase1t64:s390x (1.3.0-12build1) ... 167s Setting up libsz2:s390x (1.1.3-1) ... 167s Setting up python3-mako (1.3.5-1) ... 167s Setting up libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... 167s Setting up gdal-plugins:s390x (3.9.1+dfsg-1build1) ... 167s Setting up postgresql-16-postgis-3-scripts (3.4.2+dfsg-2) ... 167s update-alternatives: using /usr/share/postgresql/16/extension/postgis-3.control to provide /usr/share/postgresql/16/extension/postgis.control (postgresql-16-postgis.control) in auto mode 167s Setting up libodbcinst2:s390x (2.3.12-1ubuntu1) ... 167s Setting up liblapack3:s390x (3.12.0-3build1) ... 167s update-alternatives: using /usr/lib/s390x-linux-gnu/lapack/liblapack.so.3 to provide /usr/lib/s390x-linux-gnu/liblapack.so.3 (liblapack.so.3-s390x-linux-gnu) in auto mode 167s Setting up libarpack2t64:s390x (3.9.1-1.1build2) ... 167s Setting up python3-pytest (7.4.4-1) ... 168s Setting up libgeos-c1t64:s390x (3.12.2-1) ... 168s Setting up python3-cligj (0.7.2-2) ... 168s Setting up postgresql-client-16 (16.3-1) ... 168s update-alternatives: using /usr/share/postgresql/16/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 168s Setting up libsuperlu6:s390x (6.0.1+dfsg1-1build1) ... 168s Setting up libpoppler137:s390x (24.06.0-2) ... 168s Setting up python3-sqlalchemy (1.4.50+ds1-1build1) ... 169s Setting up libkmldom1t64:s390x (1.3.0-12build1) ... 169s Setting up postgresql-common (261) ... 170s 170s Creating config file /etc/postgresql-common/createcluster.conf with new version 170s Building PostgreSQL dictionaries from installed myspell/hunspell packages... 170s Removing obsolete dictionary files: 171s Created symlink '/etc/systemd/system/multi-user.target.wants/postgresql.service' → '/usr/lib/systemd/system/postgresql.service'. 171s Setting up python3-numpy (1:1.26.4+ds-10) ... 174s Setting up libjs-sphinxdoc (7.3.7-3) ... 174s Setting up libhdf4-0-alt:s390x (4.3.0-1) ... 174s Setting up librttopo1:s390x (1.1.0-3build2) ... 174s Setting up libfreexl1:s390x (2.0.0-1build2) ... 174s Setting up python3-alembic (1.13.1-4) ... 174s Setting up libarmadillo14 (1:14.0.1+dfsg-2) ... 174s Setting up postgresql-postgis-scripts (3.4.2+dfsg-2) ... 174s Setting up python3-geoalchemy2 (0.15.1-1) ... 174s Setting up libhdf5-103-1t64:s390x (1.10.10+repack-3.1ubuntu4) ... 174s Setting up libspatialite8t64:s390x (5.1.0-3build1) ... 174s Setting up libhdf5-hl-100t64:s390x (1.10.10+repack-3.1ubuntu4) ... 174s Setting up python3-shapely (2.0.5-2) ... 175s Setting up libnetcdf19t64:s390x (1:4.9.2-6) ... 175s Setting up postgresql-16 (16.3-1) ... 175s Creating new PostgreSQL cluster 16/main ... 175s /usr/lib/postgresql/16/bin/initdb -D /var/lib/postgresql/16/main --auth-local peer --auth-host scram-sha-256 --no-instructions 175s The files belonging to this database system will be owned by user "postgres". 175s This user must also own the server process. 175s 175s The database cluster will be initialized with locale "C.UTF-8". 175s The default database encoding has accordingly been set to "UTF8". 175s The default text search configuration will be set to "english". 175s 175s Data page checksums are disabled. 175s 175s fixing permissions on existing directory /var/lib/postgresql/16/main ... ok 175s creating subdirectories ... ok 175s selecting dynamic shared memory implementation ... posix 175s selecting default max_connections ... 100 175s selecting default shared_buffers ... 128MB 175s selecting default time zone ... Etc/UTC 175s creating configuration files ... ok 175s running bootstrap script ... ok 176s performing post-bootstrap initialization ... ok 176s syncing data to disk ... ok 179s Setting up libkmlengine1t64:s390x (1.3.0-12build1) ... 179s Setting up python3-snuggs (1.4.7-4) ... 179s Setting up python-geoalchemy2-doc (0.15.1-1) ... 179s Setting up libgdal35:s390x (3.9.1+dfsg-1build1) ... 179s Setting up postgresql-16-postgis-3 (3.4.2+dfsg-2) ... 179s Setting up postgis (3.4.2+dfsg-2) ... 179s Setting up python3-rasterio (1.3.10-2build1) ... 179s Setting up postgresql-postgis (3.4.2+dfsg-2) ... 179s Setting up autopkgtest-satdep (0) ... 179s Processing triggers for man-db (2.12.1-2) ... 180s Processing triggers for libc-bin (2.39-0ubuntu9) ... 183s (Reading database ... 60171 files and directories currently installed.) 183s Removing autopkgtest-satdep (0) ... 184s autopkgtest [16:10:13]: test setup-db-and-run-tests.py: [----------------------- 184s The files belonging to this database system will be owned by user "ubuntu". 184s This user must also own the server process. 184s 184s The database cluster will be initialized with this locale configuration: 184s provider: libc 184s LC_COLLATE: C.UTF-8 184s LC_CTYPE: C.UTF-8 184s LC_MESSAGES: C 184s LC_MONETARY: C.UTF-8 184s LC_NUMERIC: C.UTF-8 184s LC_TIME: C.UTF-8 184s The default database encoding has accordingly been set to "UTF8". 184s The default text search configuration will be set to "english". 184s 184s Data page checksums are disabled. 184s 184s creating directory /tmp/tmp0_z4kxl2/data ... ok 184s creating subdirectories ... ok 184s selecting dynamic shared memory implementation ... posix 184s selecting default max_connections ... 100 184s selecting default shared_buffers ... 128MB 184s selecting default time zone ... Etc/UTC 184s creating configuration files ... ok 184s running bootstrap script ... ok 185s performing post-bootstrap initialization ... ok 185s syncing data to disk ... ok 185s 185s Success. You can now start the database server using: 185s 185s /usr/lib/postgresql/16/bin/pg_ctl -D /tmp/tmp0_z4kxl2/data -l logfile start 185s 187s running initdb 187s starting server 187s ============================= test session starts ============================== 187s platform linux -- Python 3.12.4, pytest-7.4.4, pluggy-1.5.0 187s rootdir: /tmp/autopkgtest.fTasHg/build.hUU/src 187s configfile: pyproject.toml 187s testpaths: tests 187s collected 916 items / 133 deselected / 783 selected 187s 187s tests/test_alembic_migrations.py . [ 0%] 188s tests/test_functional.py ..........s. [ 1%] 188s tests/test_functional_geopackage.py sssss [ 2%] 190s tests/test_functional_postgresql.py .......................... [ 5%] 190s tests/test_functional_sqlite.py sssssssssssssssssssss [ 8%] 191s tests/gallery/test_insert_raster.py ........... [ 9%] 191s tests/gallery/test_length_at_insert.py . [ 9%] 191s tests/gallery/test_specific_compilation.py . [ 9%] 191s tests/gallery/test_summarystatsagg.py . [ 10%] 191s tests/test_alembic_migrations.py ..s [ 10%] 195s tests/test_functional.py s..F...F...F...F...F...F...F...F...F...F...F... [ 16%] 196s F...s......ss....ssss. [ 19%] 196s tests/test_functional_sqlite.py sssssssssssssssssss [ 21%] 196s tests/test_pickle.py . [ 21%] 196s tests/gallery/test_length_at_insert.py s [ 21%] 196s tests/gallery/test_orm_mapped_v2.py s [ 22%] 196s tests/gallery/test_specific_compilation.py s [ 22%] 196s tests/test_alembic_migrations.py ss [ 22%] 196s tests/test_functional.py sssssssssssssssssssssssssssssssssssssssssssssss [ 28%] 196s ssssssssssssssssssssssss [ 31%] 196s tests/test_functional_sqlite.py sssssssssss [ 32%] 196s tests/test_pickle.py s [ 33%] 196s tests/gallery/test_length_at_insert.py s [ 33%] 196s tests/gallery/test_orm_mapped_v2.py s [ 33%] 196s tests/gallery/test_specific_compilation.py s [ 33%] 196s tests/test_alembic_migrations.py ss [ 33%] 196s tests/test_functional.py sssssssssssssssssssssssssssssssssssssssssssssss [ 39%] 197s sssssssssssssssssssss [ 42%] 197s tests/test_pickle.py s [ 42%] 197s tests/gallery/test_length_at_insert.py s [ 42%] 197s tests/gallery/test_orm_mapped_v2.py s [ 42%] 197s tests/gallery/test_specific_compilation.py s [ 42%] 197s tests/test_alembic_migrations.py ss [ 43%] 197s tests/test_functional.py sssssssssssssssssssssssssssssssssssssssssssssss [ 49%] 197s ssssssssssssss [ 50%] 197s tests/test_pickle.py s [ 51%] 197s tests/gallery/test_orm_mapped_v2.py s [ 51%] 197s tests/test_comparator.py ............................................... [ 57%] 197s ...................... [ 60%] 197s tests/test_elements.py ................................................ [ 66%] 197s tests/test_functional_geopackage.py sss [ 66%] 197s tests/test_functional_postgresql.py ..... [ 67%] 197s tests/test_functional_sqlite.py ssssssssss [ 68%] 197s tests/test_functions.py ................................................ [ 74%] 198s ........................................................................ [ 83%] 198s ........................................................................ [ 92%] 198s .......... [ 94%] 198s tests/test_shape.py ..... [ 94%] 198s tests/test_types.py ...................................... [ 99%] 198s tests/gallery/test_disable_wrapping.py .. [100%] 198s 198s =================================== FAILURES =================================== 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Point] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRI...OINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT (1.5 2.5)'}, {'geom': '0000000021e61000003ff80000000000004004000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'POINT(1.5 2.5)'}, {...f74ad0; 0000000021e61000003ff80000000000004004000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRI...OINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT (1.5 2.5)'}, {'geom': '0000000021e61000003ff80000000000004004000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (33)! Full WKB type number was (33). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'POINT', wkt = '(1.5 2.5)' 198s use_floating_point = True 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRI...OINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT (1.5 2.5)'}, {'geom': '0000000021e61000003ff80000000000004004000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (33)! Full WKB type number was (33). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT (1.5 2.5)'}, {'geom': '0000000021e61000003ff80000000000004004000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-LineString] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(....5 2.5, 3 4)'}, {'geom': '0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'LINESTRING(1.5 2.5,...0000000000400400000000000040080000000000004010000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(....5 2.5, 3 4)'}, {'geom': '0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (34)! Full WKB type number was (34). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'LINESTRING', wkt = '(1.5 2.5, 3 4)' 198s use_floating_point = True 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(....5 2.5, 3 4)'}, {'geom': '0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (34)! Full WKB type number was (34). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING (1.5 2.5, 3 4)'}, {'geom': '0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Polygon] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geo...0000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'POLYGON((1.5 2.5, 3...000000000040180000000000003ff80000000000004004000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geo...0000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (35)! Full WKB type number was (35). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'POLYGON' 198s wkt = '((1.5 2.5, 3 4, 5 6, 1.5 2.5))', use_floating_point = True 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geo...0000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (35)! Full WKB type number was (35). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON ((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': '0000000023e610000000000001000000043ff8000000000000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Multi Point] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(...om': '0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'MULTIPOINT(1.5 2.5,...4004000000000000000000000140080000000000004010000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(...om': '0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (36)! Full WKB type number was (36). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'MULTIPOINT', wkt = '(1.5 2.5, 3 4)' 198s use_floating_point = True 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(...om': '0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (36)! Full WKB type number was (36). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT ((1.5 2.5), (3 4))'}, {'geom': '0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Multi LineString] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 ...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'MULTILINESTRING((1....00000000004034000000000000403e0000000000004044000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 ...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (37)! Full WKB type number was (37). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'MULTILINESTRING' 198s wkt = '((1.5 2.5, 3 4), (10 20, 30 40))', use_floating_point = True 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 ...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (37)! Full WKB type number was (37). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING ((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': '0000000025e6100000000000020000000002000000023ff800000000000040040000000000004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Multi Polygon] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPO...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'MULTIPOLYGON(((1.5 ...0000000000404e00000000000040240000000000004034000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPO...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (38)! Full WKB type number was (38). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'MULTIPOLYGON' 198s wkt = '(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))' 198s use_floating_point = True 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPO...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (38)! Full WKB type number was (38). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON (((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': '0000000026e610000000000002000000000300000001000000043ff8000000000000400400000000000040080000000000004010000000000000401400000000000040180000000000003 ... (36 characters truncated) ... 00003000000010000000440240000000000004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Point] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT (1 2)'}, {'geom': '0000000021e61000003ff00000000000004000000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'POINT(1 2)'}, {'geo...d142c0; 0000000021e61000003ff00000000000004000000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT (1 2)'}, {'geom': '0000000021e61000003ff00000000000004000000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (33)! Full WKB type number was (33). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'POINT', wkt = '(1 2)' 198s use_floating_point = False 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT (1 2)'}, {'geom': '0000000021e61000003ff00000000000004000000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (33)! Full WKB type number was (33). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT (1 2)'}, {'geom': '0000000021e61000003ff00000000000004000000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-LineString] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4...G (1 2, 3 4)'}, {'geom': '0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'LINESTRING(1 2, 3 4...0000000000400000000000000040080000000000004010000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4...G (1 2, 3 4)'}, {'geom': '0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (34)! Full WKB type number was (34). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'LINESTRING', wkt = '(1 2, 3 4)' 198s use_floating_point = False 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4...G (1 2, 3 4)'}, {'geom': '0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (34)! Full WKB type number was (34). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING (1 2, 3 4)'}, {'geom': '0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Polygon] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;P...0000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'POLYGON((1 2, 3 4, ...000000000040180000000000003ff00000000000004000000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;P...0000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (35)! Full WKB type number was (35). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'POLYGON', wkt = '((1 2, 3 4, 5 6, 1 2))' 198s use_floating_point = False 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;P...0000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (35)! Full WKB type number was (35). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON ((1 2, 3 4, 5 6, 1 2))'}, {'geom': '0000000023e610000000000001000000043ff0000000000000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Multi Point] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4...om': '0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'MULTIPOINT(1 2, 3 4...4000000000000000000000000140080000000000004010000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4...om': '0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (36)! Full WKB type number was (36). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'MULTIPOINT', wkt = '(1 2, 3 4)' 198s use_floating_point = False 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4...om': '0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (36)! Full WKB type number was (36). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT ((1 2), (3 4))'}, {'geom': '0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Multi LineString] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 4...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'MULTILINESTRING((1 ...00000000004034000000000000403e0000000000004044000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 4...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (37)! Full WKB type number was (37). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'MULTILINESTRING' 198s wkt = '((1 2, 3 4), (10 20, 30 40))', use_floating_point = False 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 4...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (37)! Full WKB type number was (37). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING ((1 2, 3 4), (10 20, 30 40))'}, {'geom': '0000000025e6100000000000020000000002000000023ff000000000000040000000000000004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Multi Polygon] _ 198s 198s self = 198s dialect = 198s constructor = > 198s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 198s parameters = ({'geom': 'MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 198s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 198s args = (, [{'geom': 'MULTIPOLYGON(((1 2,...0000000000404e00000000000040240000000000004034000000000000>}], , []) 198s kw = {'cache_hit': symbol('CACHE_MISS')} 198s branched = 198s yp = None 198s conn = 198s context = 198s cursor = , evt_handled = False 198s 198s def _execute_context( 198s self, 198s dialect, 198s constructor, 198s statement, 198s parameters, 198s execution_options, 198s *args, 198s **kw 198s ): 198s """Create an :class:`.ExecutionContext` and execute, returning 198s a :class:`_engine.CursorResult`.""" 198s 198s branched = self 198s if self.__branch_from: 198s # if this is a "branched" connection, do everything in terms 198s # of the "root" connection, *except* for .close(), which is 198s # the only feature that branching provides 198s self = self.__branch_from 198s 198s if execution_options: 198s yp = execution_options.get("yield_per", None) 198s if yp: 198s execution_options = execution_options.union( 198s {"stream_results": True, "max_row_buffer": yp} 198s ) 198s 198s try: 198s conn = self._dbapi_connection 198s if conn is None: 198s conn = self._revalidate_connection() 198s 198s context = constructor( 198s dialect, self, conn, execution_options, *args, **kw 198s ) 198s except (exc.PendingRollbackError, exc.ResourceClosedError): 198s raise 198s except BaseException as e: 198s self._handle_dbapi_exception( 198s e, util.text_type(statement), parameters, None, None 198s ) 198s 198s if ( 198s self._transaction 198s and not self._transaction.is_active 198s or ( 198s self._nested_transaction 198s and not self._nested_transaction.is_active 198s ) 198s ): 198s self._invalid_transaction() 198s 198s elif self._trans_context_manager: 198s TransactionalContext._trans_ctx_check(self) 198s 198s if self._is_future and self._transaction is None: 198s self._autobegin() 198s 198s context.pre_exec() 198s 198s if dialect.use_setinputsizes: 198s context._set_input_sizes() 198s 198s cursor, statement, parameters = ( 198s context.cursor, 198s context.statement, 198s context.parameters, 198s ) 198s 198s if not context.executemany: 198s parameters = parameters[0] 198s 198s if self._has_events or self.engine._has_events: 198s for fn in self.dispatch.before_cursor_execute: 198s statement, parameters = fn( 198s self, 198s cursor, 198s statement, 198s parameters, 198s context, 198s context.executemany, 198s ) 198s 198s if self._echo: 198s 198s self._log_info(statement) 198s 198s stats = context._get_cache_stats() 198s 198s if not self.engine.hide_parameters: 198s self._log_info( 198s "[%s] %r", 198s stats, 198s sql_util._repr_params( 198s parameters, batches=10, ismulti=context.executemany 198s ), 198s ) 198s else: 198s self._log_info( 198s "[%s] [SQL parameters hidden due to hide_parameters=True]" 198s % (stats,) 198s ) 198s 198s evt_handled = False 198s try: 198s if context.executemany: 198s if self.dialect._has_events: 198s for fn in self.dialect.dispatch.do_executemany: 198s if fn(cursor, statement, parameters, context): 198s evt_handled = True 198s break 198s if not evt_handled: 198s > self.dialect.do_executemany( 198s cursor, statement, parameters, context 198s 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E psycopg2.errors.InternalError_: Unknown WKB type (38)! Full WKB type number was (38). 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 198s 198s The above exception was the direct cause of the following exception: 198s 198s self = 198s dialect_name = 'postgresql', base = 198s conn = 198s metadata = MetaData(), geom_type = 'MULTIPOLYGON' 198s wkt = '(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))' 198s use_floating_point = False 198s 198s @pytest.mark.parametrize( 198s "geom_type,wkt", 198s [ 198s pytest.param("POINT", "(1 2)", id="Point"), 198s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 198s pytest.param("POINTM", "(1 2 3)", id="Point M"), 198s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 198s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 198s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 198s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 198s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 198s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 198s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 198s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 198s pytest.param( 198s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 198s ), 198s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 198s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 198s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 198s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 198s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 198s pytest.param( 198s "MULTILINESTRINGZ", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString Z", 198s ), 198s pytest.param( 198s "MULTILINESTRINGM", 198s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 198s id="Multi LineString M", 198s ), 198s pytest.param( 198s "MULTILINESTRINGZM", 198s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 198s id="Multi LineString ZM", 198s ), 198s pytest.param( 198s "MULTIPOLYGON", 198s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 198s id="Multi Polygon", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZ", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon Z", 198s ), 198s pytest.param( 198s "MULTIPOLYGONM", 198s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 198s id="Multi Polygon M", 198s ), 198s pytest.param( 198s "MULTIPOLYGONZM", 198s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 198s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 198s id="Multi Polygon ZM", 198s ), 198s ], 198s ) 198s @pytest.mark.parametrize( 198s "use_floating_point", 198s [ 198s pytest.param(True, id="Use floating point"), 198s pytest.param(False, id="Do not use floating point"), 198s ], 198s ) 198s def test_insert_all_geom_types( 198s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 198s ): 198s """Test insertion and selection of all geometry types.""" 198s ndims = 2 198s if "Z" in geom_type[-2:]: 198s ndims += 1 198s if geom_type.endswith("M"): 198s ndims += 1 198s has_m = True 198s else: 198s has_m = False 198s 198s if ndims > 2 and dialect_name == "mysql": 198s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 198s pytest.xfail(reason="MySQL only supports 2D geometry types") 198s 198s class GeomTypeTable(base): 198s __tablename__ = "test_geom_types" 198s id = Column(Integer, primary_key=True) 198s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 198s 198s metadata.drop_all(bind=conn, checkfirst=True) 198s metadata.create_all(bind=conn) 198s 198s if use_floating_point: 198s wkt = wkt.replace("1 2", "1.5 2.5") 198s 198s inserted_wkt = f"{geom_type}{wkt}" 198s 198s # Use the DB to generate the corresponding raw WKB 198s raw_wkb = conn.execute( 198s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 198s ).scalar() 198s 198s wkb_elem = WKBElement(raw_wkb, srid=4326) 198s inserted_elements = [ 198s {"geom": inserted_wkt}, 198s {"geom": f"SRID=4326;{inserted_wkt}"}, 198s {"geom": WKTElement(inserted_wkt, srid=4326)}, 198s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 198s ] 198s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 198s # Currently Shapely does not support geometry types with M dimension 198s inserted_elements.append({"geom": wkb_elem}) 198s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 198s 198s # Insert the elements 198s > conn.execute( 198s GeomTypeTable.__table__.insert(), 198s inserted_elements, 198s ) 198s 198s tests/test_functional.py:393: 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 198s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 198s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 198s return connection._execute_clauseelement( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 198s ret = self._execute_context( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 198s self._handle_dbapi_exception( 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 198s util.raise_( 198s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 198s raise exception 198s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 198s self.dialect.do_executemany( 198s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 198s context._psycopg2_fetched_rows = xtras.execute_values( 198s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 198s 198s cur = 198s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 198s argslist = ({'geom': 'MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 198s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 198s 198s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 198s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 198s 198s :param cur: the cursor to use to execute the query. 198s 198s :param sql: the query to execute. It must contain a single ``%s`` 198s placeholder, which will be replaced by a `VALUES list`__. 198s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 198s 198s :param argslist: sequence of sequences or dictionaries with the arguments 198s to send to the query. The type and content must be consistent with 198s *template*. 198s 198s :param template: the snippet to merge to every item in *argslist* to 198s compose the query. 198s 198s - If the *argslist* items are sequences it should contain positional 198s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 198s are constants value...). 198s 198s - If the *argslist* items are mappings it should contain named 198s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 198s 198s If not specified, assume the arguments are sequence and use a simple 198s positional template (i.e. ``(%s, %s, ...)``), with the number of 198s placeholders sniffed by the first element in *argslist*. 198s 198s :param page_size: maximum number of *argslist* items to include in every 198s statement. If there are more items the function will execute more than 198s one statement. 198s 198s :param fetch: if `!True` return the query results into a list (like in a 198s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 198s clause. 198s 198s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 198s 198s After the execution of the function the `cursor.rowcount` property will 198s **not** contain a total result. 198s 198s While :sql:`INSERT` is an obvious candidate for this function it is 198s possible to use it with other statements, for example:: 198s 198s >>> cur.execute( 198s ... "create table test (id int primary key, v1 int, v2 int)") 198s 198s >>> execute_values(cur, 198s ... "INSERT INTO test (id, v1, v2) VALUES %s", 198s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 198s 198s >>> execute_values(cur, 198s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 198s ... WHERE test.id = data.id""", 198s ... [(1, 20), (4, 50)]) 198s 198s >>> cur.execute("select * from test order by id") 198s >>> cur.fetchall() 198s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 198s 198s ''' 198s from psycopg2.sql import Composable 198s if isinstance(sql, Composable): 198s sql = sql.as_string(cur) 198s 198s # we can't just use sql % vals because vals is bytes: if sql is bytes 198s # there will be some decoding error because of stupid codec used, and Py3 198s # doesn't implement % on bytes. 198s if not isinstance(sql, bytes): 198s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 198s pre, post = _split_sql(sql) 198s 198s result = [] if fetch else None 198s for page in _paginate(argslist, page_size=page_size): 198s if template is None: 198s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 198s parts = pre[:] 198s for args in page: 198s parts.append(cur.mogrify(template, args)) 198s parts.append(b',') 198s parts[-1:] = post 198s > cur.execute(b''.join(parts)) 198s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (38)! Full WKB type number was (38). 198s E 198s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 198s E [parameters: ({'geom': 'MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON (((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': '0000000026e610000000000002000000000300000001000000043ff0000000000000400000000000000040080000000000004010000000000000401400000000000040180000000000003 ... (36 characters truncated) ... 00003000000010000000440240000000000004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'})] 198s E (Background on this error at: https://sqlalche.me/e/14/2j85) 198s 198s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 198s =============================== warnings summary =============================== 198s tests/__init__.py:9 198s /tmp/autopkgtest.fTasHg/build.hUU/src/tests/__init__.py:9: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 198s from pkg_resources import parse_version 198s 198s tests/test_functional.py::TestAdmin::test_nullable[postgresql] 198s /tmp/autopkgtest.fTasHg/build.hUU/src/geoalchemy2/types/__init__.py:191: UserWarning: srid not enforced when geometry_type is None 198s warnings.warn("srid not enforced when geometry_type is None") 198s 198s tests/test_functional.py::TestCallFunction::test_ST_Buffer[postgresql] 198s /tmp/autopkgtest.fTasHg/build.hUU/src/tests/test_functional.py:791: LegacyAPIWarning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9) 198s lake = session.query(Lake).get(lake_id) 198s 198s tests/test_functional.py::TestUpdateORM::test_other_type_fail[postgresql] 198s tests/test_pickle.py::TestPickle::test_pickle_unpickle[postgresql] 198s /tmp/autopkgtest.fTasHg/build.hUU/src/tests/conftest.py:200: SAWarning: transaction already deassociated from connection 198s trans.rollback() 198s 198s -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html 198s =========================== short test summary info ============================ 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Point] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-LineString] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Polygon] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Multi Point] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Multi LineString] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Multi Polygon] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Point] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-LineString] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Polygon] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Multi Point] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Multi LineString] 198s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Multi Polygon] 198s === 12 failed, 473 passed, 298 skipped, 133 deselected, 5 warnings in 11.67s === 198s stopping postgres server 198s postgres server error log 198s 198s 2024-07-26 16:10:14.202 UTC [5035] LOG: starting PostgreSQL 16.3 (Ubuntu 16.3-1) on s390x-ibm-linux-gnu, compiled by gcc (Ubuntu 13.2.0-25ubuntu1) 13.2.0, 64-bit 198s 2024-07-26 16:10:14.202 UTC [5035] LOG: listening on IPv6 address "::1", port 58755 198s 2024-07-26 16:10:14.202 UTC [5035] LOG: listening on IPv4 address "127.0.0.1", port 58755 198s 2024-07-26 16:10:14.202 UTC [5035] LOG: listening on Unix socket "/tmp/tmp0_z4kxl2/tmp/.s.PGSQL.58755" 198s 2024-07-26 16:10:14.205 UTC [5038] LOG: database system was shut down at 2024-07-26 16:10:14 UTC 198s 2024-07-26 16:10:14.211 UTC [5035] LOG: database system is ready to accept connections 198s 2024-07-26 16:10:16.701 UTC [5044] WARNING: there is no transaction in progress 198s 2024-07-26 16:10:16.793 UTC [5055] ERROR: null value in column "geom_not_nullable" of relation "nullable_geom_type" violates not-null constraint 198s 2024-07-26 16:10:16.793 UTC [5055] DETAIL: Failing row contains (3, null, null, 0020000002000010E600000002000000000000000000000000000000003FF000..., null). 198s 2024-07-26 16:10:16.793 UTC [5055] STATEMENT: INSERT INTO nullable_geom_type (geom_not_nullable, geom_nullable, geom_col_not_nullable, geom_col_nullable) VALUES (ST_GeomFromEWKT(NULL), ST_GeomFromEWKT(NULL), ST_GeomFromEWKT('SRID=4326;LINESTRING(0 0,1 1)'), ST_GeomFromEWKT(NULL)) RETURNING nullable_geom_type.id 198s 2024-07-26 16:10:16.793 UTC [5055] ERROR: null value in column "geom_col_not_nullable" of relation "nullable_geom_type" violates not-null constraint 198s 2024-07-26 16:10:16.793 UTC [5055] DETAIL: Failing row contains (4, 0020000002000010E600000002000000000000000000000000000000003FF000..., null, null, null). 198s 2024-07-26 16:10:16.793 UTC [5055] STATEMENT: INSERT INTO nullable_geom_type (geom_not_nullable, geom_nullable, geom_col_not_nullable, geom_col_nullable) VALUES (ST_GeomFromEWKT('SRID=4326;LINESTRING(0 0,1 1)'), ST_GeomFromEWKT(NULL), ST_GeomFromEWKT(NULL), ST_GeomFromEWKT(NULL)) RETURNING nullable_geom_type.id 198s 2024-07-26 16:10:16.996 UTC [5062] ERROR: new row for relation "contrained_lake" violates check constraint "check_geom_sk" 198s 2024-07-26 16:10:16.996 UTC [5062] DETAIL: Failing row contains (4, null, should fail, null). 198s 2024-07-26 16:10:16.996 UTC [5062] STATEMENT: INSERT INTO contrained_lake (a_str, checked_str, geom) VALUES (NULL, 'should fail', ST_GeomFromEWKT(NULL)) RETURNING contrained_lake.id 198s 2024-07-26 16:10:19.019 UTC [5074] ERROR: contains: Operation on mixed SRID geometries (Polygon, 4326) != (Point, 0) 198s 2024-07-26 16:10:19.019 UTC [5074] STATEMENT: SELECT gis.lake.id AS gis_lake_id, ST_AsEWKB(gis.lake.geom) AS gis_lake_geom 198s FROM gis.lake 198s WHERE ST_Within('POINT(0 0)', ST_Buffer(gis.lake.geom, 2)) 198s 2024-07-26 16:10:21.020 UTC [5110] ERROR: Unknown WKB type (33)! Full WKB type number was (33). 198s 2024-07-26 16:10:21.020 UTC [5110] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('POINT(1.5 2.5)')),(ST_GeomFromEWKT('SRID=4326;POINT(1.5 2.5)')),(ST_GeomFromEWKT('SRID=4326;POINT(1.5 2.5)')),(ST_GeomFromEWKT('SRID=4326;POINT(1.5 2.5)')),(ST_GeomFromEWKT('SRID=4326;POINT (1.5 2.5)')),(ST_GeomFromEWKT('0000000021e61000003ff80000000000004004000000000000')) 198s 2024-07-26 16:10:21.399 UTC [5114] ERROR: Unknown WKB type (34)! Full WKB type number was (34). 198s 2024-07-26 16:10:21.399 UTC [5114] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('LINESTRING(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING (1.5 2.5, 3 4)')),(ST_GeomFromEWKT('0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000')) 198s 2024-07-26 16:10:21.711 UTC [5118] ERROR: Unknown WKB type (35)! Full WKB type number was (35). 198s 2024-07-26 16:10:21.711 UTC [5118] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('SRID=4326;POLYGON ((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('0000000023e610000000000001000000043ff8000000000000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000')) 198s 2024-07-26 16:10:22.043 UTC [5122] ERROR: Unknown WKB type (36)! Full WKB type number was (36). 198s 2024-07-26 16:10:22.043 UTC [5122] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTIPOINT(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT ((1.5 2.5), (3 4))')),(ST_GeomFromEWKT('0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000')) 198s 2024-07-26 16:10:22.391 UTC [5126] ERROR: Unknown WKB type (37)! Full WKB type number was (37). 198s 2024-07-26 16:10:22.391 UTC [5126] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING ((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('0000000025e6100000000000020000000002000000023ff800000000000040040000000000004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000')) 198s 2024-07-26 16:10:22.770 UTC [5130] ERROR: Unknown WKB type (38)! Full WKB type number was (38). 198s 2024-07-26 16:10:22.770 UTC [5130] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON (((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('0000000026e610000000000002000000000300000001000000043ff8000000000000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff800000000000040040000000000000000000003000000010000000440240000000000004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000')) 198s 2024-07-26 16:10:23.120 UTC [5134] ERROR: Unknown WKB type (33)! Full WKB type number was (33). 198s 2024-07-26 16:10:23.120 UTC [5134] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('POINT(1 2)')),(ST_GeomFromEWKT('SRID=4326;POINT(1 2)')),(ST_GeomFromEWKT('SRID=4326;POINT(1 2)')),(ST_GeomFromEWKT('SRID=4326;POINT(1 2)')),(ST_GeomFromEWKT('SRID=4326;POINT (1 2)')),(ST_GeomFromEWKT('0000000021e61000003ff00000000000004000000000000000')) 198s 2024-07-26 16:10:23.415 UTC [5138] ERROR: Unknown WKB type (34)! Full WKB type number was (34). 198s 2024-07-26 16:10:23.415 UTC [5138] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('LINESTRING(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING (1 2, 3 4)')),(ST_GeomFromEWKT('0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000')) 198s 2024-07-26 16:10:23.722 UTC [5142] ERROR: Unknown WKB type (35)! Full WKB type number was (35). 198s 2024-07-26 16:10:23.722 UTC [5142] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('POLYGON((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('SRID=4326;POLYGON ((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('0000000023e610000000000001000000043ff0000000000000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000')) 198s 2024-07-26 16:10:24.150 UTC [5146] ERROR: Unknown WKB type (36)! Full WKB type number was (36). 198s 2024-07-26 16:10:24.150 UTC [5146] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTIPOINT(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT ((1 2), (3 4))')),(ST_GeomFromEWKT('0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000')) 198s 2024-07-26 16:10:24.489 UTC [5150] ERROR: Unknown WKB type (37)! Full WKB type number was (37). 198s 2024-07-26 16:10:24.489 UTC [5150] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTILINESTRING((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING ((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('0000000025e6100000000000020000000002000000023ff000000000000040000000000000004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000')) 198s 2024-07-26 16:10:24.928 UTC [5154] ERROR: Unknown WKB type (38)! Full WKB type number was (38). 198s 2024-07-26 16:10:24.928 UTC [5154] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON (((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('0000000026e610000000000002000000000300000001000000043ff0000000000000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff000000000000040000000000000000000000003000000010000000440240000000000004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000')) 198s 2024-07-26 16:10:25.413 UTC [5162] ERROR: function st_geomfromewkt(integer) does not exist at character 26 198s 2024-07-26 16:10:25.413 UTC [5162] HINT: No function matches the given name and argument types. You might need to add explicit type casts. 198s 2024-07-26 16:10:25.413 UTC [5162] STATEMENT: UPDATE gis.lake SET geom=ST_GeomFromEWKT(1) WHERE gis.lake.id = 1 198s 2024-07-26 16:10:25.483 UTC [5164] ERROR: function st_unknownfunction(geometry, integer) does not exist at character 8 198s 2024-07-26 16:10:25.483 UTC [5164] HINT: No function matches the given name and argument types. You might need to add explicit type casts. 198s 2024-07-26 16:10:25.483 UTC [5164] STATEMENT: SELECT ST_UnknownFunction(gis.lake.geom, 2) AS "ST_UnknownFunction_1" 198s FROM gis.lake 198s 2024-07-26 16:10:27.294 UTC [5035] LOG: received smart shutdown request 198s 2024-07-26 16:10:27.301 UTC [5035] LOG: background worker "logical replication launcher" (PID 5041) exited with exit code 1 198s 2024-07-26 16:10:27.303 UTC [5036] LOG: shutting down 198s 2024-07-26 16:10:27.303 UTC [5036] LOG: checkpoint starting: shutdown immediate 198s 2024-07-26 16:10:27.331 UTC [5036] LOG: checkpoint complete: wrote 2143 buffers (13.1%); 0 WAL file(s) added, 0 removed, 1 recycled; write=0.018 s, sync=0.001 s, total=0.029 s; sync files=0, longest=0.000 s, average=0.000 s; distance=17580 kB, estimate=17580 kB; lsn=0/261A3E0, redo lsn=0/261A3E0 198s 2024-07-26 16:10:27.343 UTC [5035] LOG: database system is shut down 198s 198s clean up 199s autopkgtest [16:10:28]: test setup-db-and-run-tests.py: -----------------------] 199s setup-db-and-run-tests.py FAIL non-zero exit status 1 199s autopkgtest [16:10:28]: test setup-db-and-run-tests.py: - - - - - - - - - - results - - - - - - - - - - 200s autopkgtest [16:10:29]: @@@@@@@@@@@@@@@@@@@@ summary 200s setup-db-and-run-tests.py FAIL non-zero exit status 1 213s nova [W] Using flock in scalingstack-bos01-s390x 213s Creating nova instance adt-oracular-s390x-geoalchemy2-20240726-160706-juju-7f2275-prod-proposed-migration-environment-3-87a53fad-b164-42ce-85a9-f063c5a8caeb from image adt/ubuntu-oracular-s390x-server-20240726.img (UUID dfbcf47e-ed98-4bf2-ad84-36ab9700c5bd)...