0s autopkgtest [10:22:00]: starting date and time: 2024-06-16 10:22:00+0000 0s autopkgtest [10:22:00]: git checkout: 433ed4cb Merge branch 'skia/nova_flock' into 'ubuntu/5.34+prod' 0s autopkgtest [10:22:00]: host juju-7f2275-prod-proposed-migration-environment-3; command line: /home/ubuntu/autopkgtest/runner/autopkgtest --output-dir /tmp/autopkgtest-work.3bv38a_1/out --timeout-copy=6000 --setup-commands /home/ubuntu/autopkgtest-cloud/worker-config-production/setup-canonical.sh --apt-pocket=proposed=src:geoalchemy2 --apt-upgrade geoalchemy2 --timeout-short=300 --timeout-copy=20000 --timeout-build=20000 --env=ADT_TEST_TRIGGERS=geoalchemy2/0.15.1-1 -- ssh -s /home/ubuntu/autopkgtest/ssh-setup/nova -- --flavor autopkgtest --security-groups autopkgtest-juju-7f2275-prod-proposed-migration-environment-3@bos01-s390x-4.secgroup --name adt-oracular-s390x-geoalchemy2-20240616-102200-juju-7f2275-prod-proposed-migration-environment-3-fb87bd89-0a0e-4544-95a3-8244b7133df3 --image adt/ubuntu-oracular-s390x-server --keyname testbed-juju-7f2275-prod-proposed-migration-environment-3 --net-id=net_prod-proposed-migration -e TERM=linux -e ''"'"'http_proxy=http://squid.internal:3128'"'"'' -e ''"'"'https_proxy=http://squid.internal:3128'"'"'' -e ''"'"'no_proxy=127.0.0.1,127.0.1.1,login.ubuntu.com,localhost,localdomain,novalocal,internal,archive.ubuntu.com,ports.ubuntu.com,security.ubuntu.com,ddebs.ubuntu.com,changelogs.ubuntu.com,keyserver.ubuntu.com,launchpadlibrarian.net,launchpadcontent.net,launchpad.net,10.24.0.0/24,keystone.ps5.canonical.com,objectstorage.prodstack5.canonical.com'"'"'' --mirror=http://us.ports.ubuntu.com/ubuntu-ports/ 146s autopkgtest [10:24:26]: testbed dpkg architecture: s390x 146s autopkgtest [10:24:26]: testbed apt version: 2.9.5 146s autopkgtest [10:24:26]: @@@@@@@@@@@@@@@@@@@@ test bed setup 147s Get:1 http://ftpmaster.internal/ubuntu oracular-proposed InRelease [110 kB] 147s Get:2 http://ftpmaster.internal/ubuntu oracular-proposed/multiverse Sources [2576 B] 147s Get:3 http://ftpmaster.internal/ubuntu oracular-proposed/main Sources [36.1 kB] 147s Get:4 http://ftpmaster.internal/ubuntu oracular-proposed/restricted Sources [7052 B] 147s Get:5 http://ftpmaster.internal/ubuntu oracular-proposed/universe Sources [389 kB] 147s Get:6 http://ftpmaster.internal/ubuntu oracular-proposed/main s390x Packages [43.9 kB] 147s Get:7 http://ftpmaster.internal/ubuntu oracular-proposed/restricted s390x Packages [1860 B] 147s Get:8 http://ftpmaster.internal/ubuntu oracular-proposed/universe s390x Packages [298 kB] 147s Get:9 http://ftpmaster.internal/ubuntu oracular-proposed/multiverse s390x Packages [2528 B] 148s Fetched 892 kB in 1s (1094 kB/s) 148s Reading package lists... 150s Reading package lists... 151s Building dependency tree... 151s Reading state information... 151s Calculating upgrade... 151s The following packages will be upgraded: 151s libldap-common libldap2 151s 2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 151s Need to get 230 kB of archives. 151s After this operation, 16.4 kB disk space will be freed. 151s Get:1 http://ftpmaster.internal/ubuntu oracular/main s390x libldap-common all 2.6.7+dfsg-1~exp1ubuntu9 [31.5 kB] 151s Get:2 http://ftpmaster.internal/ubuntu oracular/main s390x libldap2 s390x 2.6.7+dfsg-1~exp1ubuntu9 [199 kB] 152s Fetched 230 kB in 0s (537 kB/s) 152s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 54671 files and directories currently installed.) 152s Preparing to unpack .../libldap-common_2.6.7+dfsg-1~exp1ubuntu9_all.deb ... 152s Unpacking libldap-common (2.6.7+dfsg-1~exp1ubuntu9) over (2.6.7+dfsg-1~exp1ubuntu8) ... 152s Preparing to unpack .../libldap2_2.6.7+dfsg-1~exp1ubuntu9_s390x.deb ... 152s Unpacking libldap2:s390x (2.6.7+dfsg-1~exp1ubuntu9) over (2.6.7+dfsg-1~exp1ubuntu8) ... 152s Setting up libldap-common (2.6.7+dfsg-1~exp1ubuntu9) ... 152s Setting up libldap2:s390x (2.6.7+dfsg-1~exp1ubuntu9) ... 152s Processing triggers for man-db (2.12.1-2) ... 152s Processing triggers for libc-bin (2.39-0ubuntu9) ... 153s Reading package lists... 153s Building dependency tree... 153s Reading state information... 153s 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 154s Hit:1 http://ftpmaster.internal/ubuntu oracular-proposed InRelease 154s Hit:2 http://ftpmaster.internal/ubuntu oracular InRelease 154s Hit:3 http://ftpmaster.internal/ubuntu oracular-updates InRelease 154s Hit:4 http://ftpmaster.internal/ubuntu oracular-security InRelease 155s Reading package lists... 155s Reading package lists... 155s Building dependency tree... 155s Reading state information... 156s Calculating upgrade... 156s 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 156s Reading package lists... 156s Building dependency tree... 156s Reading state information... 156s 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. 159s autopkgtest [10:24:39]: testbed running kernel: Linux 6.8.0-31-generic #31-Ubuntu SMP Sat Apr 20 00:14:26 UTC 2024 159s autopkgtest [10:24:39]: @@@@@@@@@@@@@@@@@@@@ apt-source geoalchemy2 162s Get:1 http://ftpmaster.internal/ubuntu oracular-proposed/universe geoalchemy2 0.15.1-1 (dsc) [2597 B] 162s Get:2 http://ftpmaster.internal/ubuntu oracular-proposed/universe geoalchemy2 0.15.1-1 (tar) [219 kB] 162s Get:3 http://ftpmaster.internal/ubuntu oracular-proposed/universe geoalchemy2 0.15.1-1 (diff) [8328 B] 162s gpgv: Signature made Sun May 19 11:34:07 2024 UTC 162s gpgv: using RSA key FB8ACFA78C726089C38AD0269605A1098C63B92A 162s gpgv: Can't check signature: No public key 162s dpkg-source: warning: cannot verify inline signature for ./geoalchemy2_0.15.1-1.dsc: no acceptable signature found 162s autopkgtest [10:24:42]: testing package geoalchemy2 version 0.15.1-1 162s autopkgtest [10:24:42]: build not needed 163s autopkgtest [10:24:43]: test setup-db-and-run-tests.py: preparing testbed 164s Reading package lists... 164s Building dependency tree... 164s Reading state information... 164s Starting pkgProblemResolver with broken count: 0 164s Starting 2 pkgProblemResolver with broken count: 0 164s Done 164s The following additional packages will be installed: 164s gdal-data gdal-plugins libaec0 libarmadillo12 libarpack2t64 libblas3 164s libblosc1 libboost-serialization1.83.0 libcfitsio10t64 libfreexl1 164s libfyba0t64 libgdal35 libgeos-c1t64 libgeos3.12.2 libgeotiff5 libgfortran5 164s libgif7 libgmpxx4ldbl libhdf4-0-alt libhdf5-103-1t64 libhdf5-hl-100t64 164s libjs-jquery libjs-sphinxdoc libjs-underscore libjson-perl libkmlbase1t64 164s libkmldom1t64 libkmlengine1t64 liblapack3 liblcms2-2 libltdl7 libminizip1t64 164s libmysqlclient21 libnetcdf19t64 libodbc2 libodbcinst2 libogdi4.1 164s libopenjp2-7 libpoppler134 libpq5 libproj25 libqhull-r8.0 librttopo1 164s libsfcgal1t64 libsnappy1v5 libspatialite8t64 libsuperlu6 libsz2 164s liburiparser1 libxerces-c3.2t64 libxslt1.1 mysql-common postgis 164s postgresql-16 postgresql-16-postgis-3 postgresql-16-postgis-3-scripts 164s postgresql-client-16 postgresql-client-common postgresql-common 164s postgresql-postgis postgresql-postgis-scripts proj-data 164s python-geoalchemy2-doc python3-affine python3-alembic python3-click 164s python3-click-plugins python3-cligj python3-colorama python3-geoalchemy2 164s python3-greenlet python3-iniconfig python3-mako python3-mysqldb 164s python3-numpy python3-packaging python3-pluggy python3-psycopg2 164s python3-pytest python3-rasterio python3-shapely python3-snuggs 164s python3-sqlalchemy ssl-cert unixodbc-common 164s Suggested packages: 164s geotiff-bin gdal-bin libgeotiff-epsg libhdf4-alt-dev hdf4-tools 164s liblcms2-utils odbc-postgresql tdsodbc ogdi-bin proj-bin postgresql-doc-16 164s python-greenlet-dev python-greenlet-doc python-mako-doc python3-beaker 164s default-mysql-server | virtual-mysql-server gcc gfortran python-numpy-doc 164s python3-dev python-psycopg2-doc python3-matplotlib python-sqlalchemy-doc 164s python3-fdb python3-pymssql python3-asyncpg python3-aiosqlite 164s Recommended packages: 164s proj-bin javascript-common libjson-xs-perl poppler-data postgis-doc 164s python3-sqlalchemy-ext 165s The following NEW packages will be installed: 165s autopkgtest-satdep gdal-data gdal-plugins libaec0 libarmadillo12 165s libarpack2t64 libblas3 libblosc1 libboost-serialization1.83.0 165s libcfitsio10t64 libfreexl1 libfyba0t64 libgdal35 libgeos-c1t64 libgeos3.12.2 165s libgeotiff5 libgfortran5 libgif7 libgmpxx4ldbl libhdf4-0-alt 165s libhdf5-103-1t64 libhdf5-hl-100t64 libjs-jquery libjs-sphinxdoc 165s libjs-underscore libjson-perl libkmlbase1t64 libkmldom1t64 libkmlengine1t64 165s liblapack3 liblcms2-2 libltdl7 libminizip1t64 libmysqlclient21 165s libnetcdf19t64 libodbc2 libodbcinst2 libogdi4.1 libopenjp2-7 libpoppler134 165s libpq5 libproj25 libqhull-r8.0 librttopo1 libsfcgal1t64 libsnappy1v5 165s libspatialite8t64 libsuperlu6 libsz2 liburiparser1 libxerces-c3.2t64 165s libxslt1.1 mysql-common postgis postgresql-16 postgresql-16-postgis-3 165s postgresql-16-postgis-3-scripts postgresql-client-16 165s postgresql-client-common postgresql-common postgresql-postgis 165s postgresql-postgis-scripts proj-data python-geoalchemy2-doc python3-affine 165s python3-alembic python3-click python3-click-plugins python3-cligj 165s python3-colorama python3-geoalchemy2 python3-greenlet python3-iniconfig 165s python3-mako python3-mysqldb python3-numpy python3-packaging python3-pluggy 165s python3-psycopg2 python3-pytest python3-rasterio python3-shapely 165s python3-snuggs python3-sqlalchemy ssl-cert unixodbc-common 165s 0 upgraded, 86 newly installed, 0 to remove and 0 not upgraded. 165s Need to get 69.5 MB/69.5 MB of archives. 165s After this operation, 279 MB of additional disk space will be used. 165s Get:1 /tmp/autopkgtest.k0pZv7/1-autopkgtest-satdep.deb autopkgtest-satdep s390x 0 [788 B] 165s Get:2 http://ftpmaster.internal/ubuntu oracular/main s390x libjson-perl all 4.10000-1 [81.9 kB] 165s Get:3 http://ftpmaster.internal/ubuntu oracular/main s390x postgresql-client-common all 260 [36.4 kB] 165s Get:4 http://ftpmaster.internal/ubuntu oracular/main s390x ssl-cert all 1.1.2ubuntu1 [17.8 kB] 165s Get:5 http://ftpmaster.internal/ubuntu oracular/main s390x postgresql-common all 260 [162 kB] 165s Get:6 http://ftpmaster.internal/ubuntu oracular/universe s390x gdal-data all 3.9.0+dfsg-1 [268 kB] 165s Get:7 http://ftpmaster.internal/ubuntu oracular/universe s390x gdal-plugins s390x 3.9.0+dfsg-1 [25.1 kB] 165s Get:8 http://ftpmaster.internal/ubuntu oracular/universe s390x libaec0 s390x 1.1.3-1 [25.7 kB] 165s Get:9 http://ftpmaster.internal/ubuntu oracular/main s390x libblas3 s390x 3.12.0-3build1 [245 kB] 165s Get:10 http://ftpmaster.internal/ubuntu oracular/main s390x libgfortran5 s390x 14.1.0-1ubuntu1 [600 kB] 166s Get:11 http://ftpmaster.internal/ubuntu oracular/main s390x liblapack3 s390x 3.12.0-3build1 [2976 kB] 166s Get:12 http://ftpmaster.internal/ubuntu oracular/universe s390x libarpack2t64 s390x 3.9.1-1.1build2 [113 kB] 166s Get:13 http://ftpmaster.internal/ubuntu oracular/universe s390x libsuperlu6 s390x 6.0.1+dfsg1-1build1 [232 kB] 166s Get:14 http://ftpmaster.internal/ubuntu oracular/universe s390x libarmadillo12 s390x 1:12.8.2+dfsg-1 [110 kB] 166s Get:15 http://ftpmaster.internal/ubuntu oracular/main s390x libsnappy1v5 s390x 1.2.1-1 [33.0 kB] 166s Get:16 http://ftpmaster.internal/ubuntu oracular/universe s390x libblosc1 s390x 1.21.5+ds-1build1 [27.7 kB] 166s Get:17 http://ftpmaster.internal/ubuntu oracular/main s390x libboost-serialization1.83.0 s390x 1.83.0-3ubuntu1 [348 kB] 167s Get:18 http://ftpmaster.internal/ubuntu oracular/universe s390x libcfitsio10t64 s390x 4.3.1-1.1build2 [609 kB] 167s Get:19 http://ftpmaster.internal/ubuntu oracular/universe s390x libfyba0t64 s390x 4.1.1-11build1 [125 kB] 167s Get:20 http://ftpmaster.internal/ubuntu oracular/universe s390x libminizip1t64 s390x 1:1.3.dfsg-3.1ubuntu2 [24.4 kB] 167s Get:21 http://ftpmaster.internal/ubuntu oracular/universe s390x libfreexl1 s390x 2.0.0-1build2 [46.5 kB] 167s Get:22 http://ftpmaster.internal/ubuntu oracular/universe s390x libgeos3.12.2 s390x 3.12.2-1 [883 kB] 167s Get:23 http://ftpmaster.internal/ubuntu oracular/universe s390x libgeos-c1t64 s390x 3.12.2-1 [90.8 kB] 167s Get:24 http://ftpmaster.internal/ubuntu oracular/universe s390x proj-data all 9.4.1-1 [7892 kB] 168s Get:25 http://ftpmaster.internal/ubuntu oracular/universe s390x libproj25 s390x 9.4.1-1 [1398 kB] 168s Get:26 http://ftpmaster.internal/ubuntu oracular/universe s390x libgeotiff5 s390x 1.7.3-1 [66.4 kB] 168s Get:27 http://ftpmaster.internal/ubuntu oracular/main s390x libgif7 s390x 5.2.2-1ubuntu1 [38.0 kB] 168s Get:28 http://ftpmaster.internal/ubuntu oracular/universe s390x libsz2 s390x 1.1.3-1 [5442 B] 168s Get:29 http://ftpmaster.internal/ubuntu oracular/universe s390x libhdf4-0-alt s390x 4.3.0-1 [300 kB] 168s Get:30 http://ftpmaster.internal/ubuntu oracular/universe s390x libhdf5-103-1t64 s390x 1.10.10+repack-3.1ubuntu4 [1436 kB] 168s Get:31 http://ftpmaster.internal/ubuntu oracular/universe s390x liburiparser1 s390x 0.9.7+dfsg-2build1 [42.1 kB] 168s Get:32 http://ftpmaster.internal/ubuntu oracular/universe s390x libkmlbase1t64 s390x 1.3.0-12build1 [50.5 kB] 168s Get:33 http://ftpmaster.internal/ubuntu oracular/universe s390x libkmldom1t64 s390x 1.3.0-12build1 [151 kB] 168s Get:34 http://ftpmaster.internal/ubuntu oracular/universe s390x libkmlengine1t64 s390x 1.3.0-12build1 [68.9 kB] 168s Get:35 http://ftpmaster.internal/ubuntu oracular/main s390x mysql-common all 5.8+1.1.0build1 [6746 B] 168s Get:36 http://ftpmaster.internal/ubuntu oracular/main s390x libmysqlclient21 s390x 8.0.37-1 [1296 kB] 168s Get:37 http://ftpmaster.internal/ubuntu oracular/universe s390x libhdf5-hl-100t64 s390x 1.10.10+repack-3.1ubuntu4 [58.0 kB] 168s Get:38 http://ftpmaster.internal/ubuntu oracular/universe s390x libnetcdf19t64 s390x 1:4.9.2-6 [498 kB] 168s Get:39 http://ftpmaster.internal/ubuntu oracular/main s390x libltdl7 s390x 2.4.7-7build1 [41.8 kB] 168s Get:40 http://ftpmaster.internal/ubuntu oracular/main s390x libodbc2 s390x 2.3.12-1ubuntu1 [162 kB] 168s Get:41 http://ftpmaster.internal/ubuntu oracular/main s390x unixodbc-common all 2.3.12-1ubuntu1 [8796 B] 168s Get:42 http://ftpmaster.internal/ubuntu oracular/main s390x libodbcinst2 s390x 2.3.12-1ubuntu1 [32.0 kB] 168s Get:43 http://ftpmaster.internal/ubuntu oracular/universe s390x libogdi4.1 s390x 4.1.1+ds-4 [220 kB] 168s Get:44 http://ftpmaster.internal/ubuntu oracular/main s390x libopenjp2-7 s390x 2.5.0-2build3 [192 kB] 168s Get:45 http://ftpmaster.internal/ubuntu oracular/main s390x liblcms2-2 s390x 2.14-2build1 [172 kB] 168s Get:46 http://ftpmaster.internal/ubuntu oracular/main s390x libpoppler134 s390x 24.02.0-1ubuntu10 [1158 kB] 168s Get:47 http://ftpmaster.internal/ubuntu oracular/main s390x libpq5 s390x 16.3-1 [144 kB] 168s Get:48 http://ftpmaster.internal/ubuntu oracular/universe s390x libqhull-r8.0 s390x 2020.2-6build1 [199 kB] 168s Get:49 http://ftpmaster.internal/ubuntu oracular/universe s390x librttopo1 s390x 1.1.0-3build2 [200 kB] 168s Get:50 http://ftpmaster.internal/ubuntu oracular/universe s390x libspatialite8t64 s390x 5.1.0-3build1 [2050 kB] 168s Get:51 http://ftpmaster.internal/ubuntu oracular/universe s390x libxerces-c3.2t64 s390x 3.2.4+debian-1.3 [942 kB] 168s Get:52 http://ftpmaster.internal/ubuntu oracular/universe s390x libgdal35 s390x 3.9.0+dfsg-1 [9055 kB] 169s Get:53 http://ftpmaster.internal/ubuntu oracular/main s390x libgmpxx4ldbl s390x 2:6.3.0+dfsg-2ubuntu6 [10.0 kB] 169s Get:54 http://ftpmaster.internal/ubuntu oracular/main s390x libjs-jquery all 3.6.1+dfsg+~3.5.14-1 [328 kB] 169s Get:55 http://ftpmaster.internal/ubuntu oracular/main s390x libjs-underscore all 1.13.4~dfsg+~1.11.4-3 [118 kB] 169s Get:56 http://ftpmaster.internal/ubuntu oracular/main s390x libjs-sphinxdoc all 7.2.6-8 [150 kB] 169s Get:57 http://ftpmaster.internal/ubuntu oracular/universe s390x libsfcgal1t64 s390x 1.5.1-3build1 [2746 kB] 169s Get:58 http://ftpmaster.internal/ubuntu oracular/main s390x libxslt1.1 s390x 1.1.39-0exp1build1 [170 kB] 169s Get:59 http://ftpmaster.internal/ubuntu oracular/universe s390x postgis s390x 3.4.2+dfsg-1ubuntu4 [168 kB] 169s Get:60 http://ftpmaster.internal/ubuntu oracular/main s390x postgresql-client-16 s390x 16.3-1 [1290 kB] 169s Get:61 http://ftpmaster.internal/ubuntu oracular/main s390x postgresql-16 s390x 16.3-1 [16.7 MB] 170s Get:62 http://ftpmaster.internal/ubuntu oracular/universe s390x postgresql-16-postgis-3-scripts all 3.4.2+dfsg-1ubuntu4 [998 kB] 170s Get:63 http://ftpmaster.internal/ubuntu oracular/universe s390x postgresql-16-postgis-3 s390x 3.4.2+dfsg-1ubuntu4 [4057 kB] 171s Get:64 http://ftpmaster.internal/ubuntu oracular/universe s390x postgresql-postgis-scripts all 3.4.2+dfsg-1ubuntu4 [14.4 kB] 171s Get:65 http://ftpmaster.internal/ubuntu oracular/universe s390x postgresql-postgis s390x 3.4.2+dfsg-1ubuntu4 [14.4 kB] 171s Get:66 http://ftpmaster.internal/ubuntu oracular-proposed/universe s390x python-geoalchemy2-doc all 0.15.1-1 [272 kB] 171s Get:67 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-affine all 2.4.0-1 [14.3 kB] 171s Get:68 http://ftpmaster.internal/ubuntu oracular/main s390x python3-mako all 1.3.5-1 [63.4 kB] 171s Get:69 http://ftpmaster.internal/ubuntu oracular/main s390x python3-greenlet s390x 3.0.3-0ubuntu5 [156 kB] 171s Get:70 http://ftpmaster.internal/ubuntu oracular/main s390x python3-sqlalchemy all 1.4.50+ds1-1build1 [1020 kB] 171s Get:71 http://ftpmaster.internal/ubuntu oracular/main s390x python3-alembic all 1.13.1-4 [149 kB] 171s Get:72 http://ftpmaster.internal/ubuntu oracular/main s390x python3-colorama all 0.4.6-4 [32.1 kB] 171s Get:73 http://ftpmaster.internal/ubuntu oracular/main s390x python3-click all 8.1.7-1 [79.1 kB] 171s Get:74 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-click-plugins all 1.1.1-4 [9288 B] 171s Get:75 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-cligj all 0.7.2-2 [7788 B] 171s Get:76 http://ftpmaster.internal/ubuntu oracular/main s390x python3-packaging all 24.0-1 [41.1 kB] 171s Get:77 http://ftpmaster.internal/ubuntu oracular-proposed/universe s390x python3-geoalchemy2 all 0.15.1-1 [51.1 kB] 171s Get:78 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-iniconfig all 1.1.1-2 [6024 B] 171s Get:79 http://ftpmaster.internal/ubuntu oracular/main s390x python3-mysqldb s390x 1.4.6-2build3 [47.5 kB] 171s Get:80 http://ftpmaster.internal/ubuntu oracular/main s390x python3-numpy s390x 1:1.26.4+ds-10 [4090 kB] 171s Get:81 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-pluggy all 1.5.0-1 [21.0 kB] 171s Get:82 http://ftpmaster.internal/ubuntu oracular/main s390x python3-psycopg2 s390x 2.9.9-1build1 [133 kB] 171s Get:83 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-pytest all 7.4.4-1 [305 kB] 171s Get:84 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-snuggs all 1.4.7-4 [6456 B] 171s Get:85 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-rasterio s390x 1.3.10-2build1 [1005 kB] 171s Get:86 http://ftpmaster.internal/ubuntu oracular/universe s390x python3-shapely s390x 2.0.4-1 [310 kB] 171s Preconfiguring packages ... 172s Fetched 69.5 MB in 6s (10.8 MB/s) 172s Selecting previously unselected package libjson-perl. 172s (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 54671 files and directories currently installed.) 172s Preparing to unpack .../00-libjson-perl_4.10000-1_all.deb ... 172s Unpacking libjson-perl (4.10000-1) ... 172s Selecting previously unselected package postgresql-client-common. 172s Preparing to unpack .../01-postgresql-client-common_260_all.deb ... 172s Unpacking postgresql-client-common (260) ... 172s Selecting previously unselected package ssl-cert. 172s Preparing to unpack .../02-ssl-cert_1.1.2ubuntu1_all.deb ... 172s Unpacking ssl-cert (1.1.2ubuntu1) ... 172s Selecting previously unselected package postgresql-common. 172s Preparing to unpack .../03-postgresql-common_260_all.deb ... 172s Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 172s Unpacking postgresql-common (260) ... 172s Selecting previously unselected package gdal-data. 172s Preparing to unpack .../04-gdal-data_3.9.0+dfsg-1_all.deb ... 172s Unpacking gdal-data (3.9.0+dfsg-1) ... 172s Selecting previously unselected package gdal-plugins:s390x. 172s Preparing to unpack .../05-gdal-plugins_3.9.0+dfsg-1_s390x.deb ... 172s Unpacking gdal-plugins:s390x (3.9.0+dfsg-1) ... 172s Selecting previously unselected package libaec0:s390x. 172s Preparing to unpack .../06-libaec0_1.1.3-1_s390x.deb ... 172s Unpacking libaec0:s390x (1.1.3-1) ... 172s Selecting previously unselected package libblas3:s390x. 172s Preparing to unpack .../07-libblas3_3.12.0-3build1_s390x.deb ... 172s Unpacking libblas3:s390x (3.12.0-3build1) ... 172s Selecting previously unselected package libgfortran5:s390x. 172s Preparing to unpack .../08-libgfortran5_14.1.0-1ubuntu1_s390x.deb ... 172s Unpacking libgfortran5:s390x (14.1.0-1ubuntu1) ... 172s Selecting previously unselected package liblapack3:s390x. 172s Preparing to unpack .../09-liblapack3_3.12.0-3build1_s390x.deb ... 172s Unpacking liblapack3:s390x (3.12.0-3build1) ... 172s Selecting previously unselected package libarpack2t64:s390x. 172s Preparing to unpack .../10-libarpack2t64_3.9.1-1.1build2_s390x.deb ... 172s Unpacking libarpack2t64:s390x (3.9.1-1.1build2) ... 172s Selecting previously unselected package libsuperlu6:s390x. 172s Preparing to unpack .../11-libsuperlu6_6.0.1+dfsg1-1build1_s390x.deb ... 172s Unpacking libsuperlu6:s390x (6.0.1+dfsg1-1build1) ... 172s Selecting previously unselected package libarmadillo12. 172s Preparing to unpack .../12-libarmadillo12_1%3a12.8.2+dfsg-1_s390x.deb ... 172s Unpacking libarmadillo12 (1:12.8.2+dfsg-1) ... 172s Selecting previously unselected package libsnappy1v5:s390x. 172s Preparing to unpack .../13-libsnappy1v5_1.2.1-1_s390x.deb ... 172s Unpacking libsnappy1v5:s390x (1.2.1-1) ... 172s Selecting previously unselected package libblosc1:s390x. 172s Preparing to unpack .../14-libblosc1_1.21.5+ds-1build1_s390x.deb ... 172s Unpacking libblosc1:s390x (1.21.5+ds-1build1) ... 172s Selecting previously unselected package libboost-serialization1.83.0:s390x. 172s Preparing to unpack .../15-libboost-serialization1.83.0_1.83.0-3ubuntu1_s390x.deb ... 172s Unpacking libboost-serialization1.83.0:s390x (1.83.0-3ubuntu1) ... 172s Selecting previously unselected package libcfitsio10t64:s390x. 172s Preparing to unpack .../16-libcfitsio10t64_4.3.1-1.1build2_s390x.deb ... 172s Unpacking libcfitsio10t64:s390x (4.3.1-1.1build2) ... 172s Selecting previously unselected package libfyba0t64:s390x. 172s Preparing to unpack .../17-libfyba0t64_4.1.1-11build1_s390x.deb ... 172s Unpacking libfyba0t64:s390x (4.1.1-11build1) ... 172s Selecting previously unselected package libminizip1t64:s390x. 172s Preparing to unpack .../18-libminizip1t64_1%3a1.3.dfsg-3.1ubuntu2_s390x.deb ... 172s Unpacking libminizip1t64:s390x (1:1.3.dfsg-3.1ubuntu2) ... 172s Selecting previously unselected package libfreexl1:s390x. 172s Preparing to unpack .../19-libfreexl1_2.0.0-1build2_s390x.deb ... 172s Unpacking libfreexl1:s390x (2.0.0-1build2) ... 172s Selecting previously unselected package libgeos3.12.2:s390x. 172s Preparing to unpack .../20-libgeos3.12.2_3.12.2-1_s390x.deb ... 172s Unpacking libgeos3.12.2:s390x (3.12.2-1) ... 172s Selecting previously unselected package libgeos-c1t64:s390x. 172s Preparing to unpack .../21-libgeos-c1t64_3.12.2-1_s390x.deb ... 172s Unpacking libgeos-c1t64:s390x (3.12.2-1) ... 172s Selecting previously unselected package proj-data. 172s Preparing to unpack .../22-proj-data_9.4.1-1_all.deb ... 172s Unpacking proj-data (9.4.1-1) ... 172s Selecting previously unselected package libproj25:s390x. 172s Preparing to unpack .../23-libproj25_9.4.1-1_s390x.deb ... 172s Unpacking libproj25:s390x (9.4.1-1) ... 172s Selecting previously unselected package libgeotiff5:s390x. 172s Preparing to unpack .../24-libgeotiff5_1.7.3-1_s390x.deb ... 172s Unpacking libgeotiff5:s390x (1.7.3-1) ... 173s Selecting previously unselected package libgif7:s390x. 173s Preparing to unpack .../25-libgif7_5.2.2-1ubuntu1_s390x.deb ... 173s Unpacking libgif7:s390x (5.2.2-1ubuntu1) ... 173s Selecting previously unselected package libsz2:s390x. 173s Preparing to unpack .../26-libsz2_1.1.3-1_s390x.deb ... 173s Unpacking libsz2:s390x (1.1.3-1) ... 173s Selecting previously unselected package libhdf4-0-alt:s390x. 173s Preparing to unpack .../27-libhdf4-0-alt_4.3.0-1_s390x.deb ... 173s Unpacking libhdf4-0-alt:s390x (4.3.0-1) ... 173s Selecting previously unselected package libhdf5-103-1t64:s390x. 173s Preparing to unpack .../28-libhdf5-103-1t64_1.10.10+repack-3.1ubuntu4_s390x.deb ... 173s Unpacking libhdf5-103-1t64:s390x (1.10.10+repack-3.1ubuntu4) ... 173s Selecting previously unselected package liburiparser1:s390x. 173s Preparing to unpack .../29-liburiparser1_0.9.7+dfsg-2build1_s390x.deb ... 173s Unpacking liburiparser1:s390x (0.9.7+dfsg-2build1) ... 173s Selecting previously unselected package libkmlbase1t64:s390x. 173s Preparing to unpack .../30-libkmlbase1t64_1.3.0-12build1_s390x.deb ... 173s Unpacking libkmlbase1t64:s390x (1.3.0-12build1) ... 173s Selecting previously unselected package libkmldom1t64:s390x. 173s Preparing to unpack .../31-libkmldom1t64_1.3.0-12build1_s390x.deb ... 173s Unpacking libkmldom1t64:s390x (1.3.0-12build1) ... 173s Selecting previously unselected package libkmlengine1t64:s390x. 173s Preparing to unpack .../32-libkmlengine1t64_1.3.0-12build1_s390x.deb ... 173s Unpacking libkmlengine1t64:s390x (1.3.0-12build1) ... 173s Selecting previously unselected package mysql-common. 173s Preparing to unpack .../33-mysql-common_5.8+1.1.0build1_all.deb ... 173s Unpacking mysql-common (5.8+1.1.0build1) ... 173s Selecting previously unselected package libmysqlclient21:s390x. 173s Preparing to unpack .../34-libmysqlclient21_8.0.37-1_s390x.deb ... 173s Unpacking libmysqlclient21:s390x (8.0.37-1) ... 173s Selecting previously unselected package libhdf5-hl-100t64:s390x. 173s Preparing to unpack .../35-libhdf5-hl-100t64_1.10.10+repack-3.1ubuntu4_s390x.deb ... 173s Unpacking libhdf5-hl-100t64:s390x (1.10.10+repack-3.1ubuntu4) ... 173s Selecting previously unselected package libnetcdf19t64:s390x. 173s Preparing to unpack .../36-libnetcdf19t64_1%3a4.9.2-6_s390x.deb ... 173s Unpacking libnetcdf19t64:s390x (1:4.9.2-6) ... 173s Selecting previously unselected package libltdl7:s390x. 173s Preparing to unpack .../37-libltdl7_2.4.7-7build1_s390x.deb ... 173s Unpacking libltdl7:s390x (2.4.7-7build1) ... 173s Selecting previously unselected package libodbc2:s390x. 173s Preparing to unpack .../38-libodbc2_2.3.12-1ubuntu1_s390x.deb ... 173s Unpacking libodbc2:s390x (2.3.12-1ubuntu1) ... 173s Selecting previously unselected package unixodbc-common. 173s Preparing to unpack .../39-unixodbc-common_2.3.12-1ubuntu1_all.deb ... 173s Unpacking unixodbc-common (2.3.12-1ubuntu1) ... 173s Selecting previously unselected package libodbcinst2:s390x. 173s Preparing to unpack .../40-libodbcinst2_2.3.12-1ubuntu1_s390x.deb ... 173s Unpacking libodbcinst2:s390x (2.3.12-1ubuntu1) ... 173s Selecting previously unselected package libogdi4.1:s390x. 173s Preparing to unpack .../41-libogdi4.1_4.1.1+ds-4_s390x.deb ... 173s Unpacking libogdi4.1:s390x (4.1.1+ds-4) ... 173s Selecting previously unselected package libopenjp2-7:s390x. 173s Preparing to unpack .../42-libopenjp2-7_2.5.0-2build3_s390x.deb ... 173s Unpacking libopenjp2-7:s390x (2.5.0-2build3) ... 173s Selecting previously unselected package liblcms2-2:s390x. 173s Preparing to unpack .../43-liblcms2-2_2.14-2build1_s390x.deb ... 173s Unpacking liblcms2-2:s390x (2.14-2build1) ... 173s Selecting previously unselected package libpoppler134:s390x. 173s Preparing to unpack .../44-libpoppler134_24.02.0-1ubuntu10_s390x.deb ... 173s Unpacking libpoppler134:s390x (24.02.0-1ubuntu10) ... 173s Selecting previously unselected package libpq5:s390x. 173s Preparing to unpack .../45-libpq5_16.3-1_s390x.deb ... 173s Unpacking libpq5:s390x (16.3-1) ... 173s Selecting previously unselected package libqhull-r8.0:s390x. 173s Preparing to unpack .../46-libqhull-r8.0_2020.2-6build1_s390x.deb ... 173s Unpacking libqhull-r8.0:s390x (2020.2-6build1) ... 173s Selecting previously unselected package librttopo1:s390x. 173s Preparing to unpack .../47-librttopo1_1.1.0-3build2_s390x.deb ... 173s Unpacking librttopo1:s390x (1.1.0-3build2) ... 173s Selecting previously unselected package libspatialite8t64:s390x. 173s Preparing to unpack .../48-libspatialite8t64_5.1.0-3build1_s390x.deb ... 173s Unpacking libspatialite8t64:s390x (5.1.0-3build1) ... 173s Selecting previously unselected package libxerces-c3.2t64:s390x. 173s Preparing to unpack .../49-libxerces-c3.2t64_3.2.4+debian-1.3_s390x.deb ... 173s Unpacking libxerces-c3.2t64:s390x (3.2.4+debian-1.3) ... 173s Selecting previously unselected package libgdal35:s390x. 173s Preparing to unpack .../50-libgdal35_3.9.0+dfsg-1_s390x.deb ... 173s Unpacking libgdal35:s390x (3.9.0+dfsg-1) ... 174s Selecting previously unselected package libgmpxx4ldbl:s390x. 174s Preparing to unpack .../51-libgmpxx4ldbl_2%3a6.3.0+dfsg-2ubuntu6_s390x.deb ... 174s Unpacking libgmpxx4ldbl:s390x (2:6.3.0+dfsg-2ubuntu6) ... 174s Selecting previously unselected package libjs-jquery. 174s Preparing to unpack .../52-libjs-jquery_3.6.1+dfsg+~3.5.14-1_all.deb ... 174s Unpacking libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... 174s Selecting previously unselected package libjs-underscore. 174s Preparing to unpack .../53-libjs-underscore_1.13.4~dfsg+~1.11.4-3_all.deb ... 174s Unpacking libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... 174s Selecting previously unselected package libjs-sphinxdoc. 174s Preparing to unpack .../54-libjs-sphinxdoc_7.2.6-8_all.deb ... 174s Unpacking libjs-sphinxdoc (7.2.6-8) ... 174s Selecting previously unselected package libsfcgal1t64. 174s Preparing to unpack .../55-libsfcgal1t64_1.5.1-3build1_s390x.deb ... 174s Unpacking libsfcgal1t64 (1.5.1-3build1) ... 174s Selecting previously unselected package libxslt1.1:s390x. 174s Preparing to unpack .../56-libxslt1.1_1.1.39-0exp1build1_s390x.deb ... 174s Unpacking libxslt1.1:s390x (1.1.39-0exp1build1) ... 174s Selecting previously unselected package postgis. 174s Preparing to unpack .../57-postgis_3.4.2+dfsg-1ubuntu4_s390x.deb ... 174s Unpacking postgis (3.4.2+dfsg-1ubuntu4) ... 174s Selecting previously unselected package postgresql-client-16. 174s Preparing to unpack .../58-postgresql-client-16_16.3-1_s390x.deb ... 174s Unpacking postgresql-client-16 (16.3-1) ... 174s Selecting previously unselected package postgresql-16. 174s Preparing to unpack .../59-postgresql-16_16.3-1_s390x.deb ... 174s Unpacking postgresql-16 (16.3-1) ... 175s Selecting previously unselected package postgresql-16-postgis-3-scripts. 175s Preparing to unpack .../60-postgresql-16-postgis-3-scripts_3.4.2+dfsg-1ubuntu4_all.deb ... 175s Unpacking postgresql-16-postgis-3-scripts (3.4.2+dfsg-1ubuntu4) ... 175s Selecting previously unselected package postgresql-16-postgis-3. 175s Preparing to unpack .../61-postgresql-16-postgis-3_3.4.2+dfsg-1ubuntu4_s390x.deb ... 175s Unpacking postgresql-16-postgis-3 (3.4.2+dfsg-1ubuntu4) ... 175s Selecting previously unselected package postgresql-postgis-scripts. 175s Preparing to unpack .../62-postgresql-postgis-scripts_3.4.2+dfsg-1ubuntu4_all.deb ... 175s Unpacking postgresql-postgis-scripts (3.4.2+dfsg-1ubuntu4) ... 175s Selecting previously unselected package postgresql-postgis. 175s Preparing to unpack .../63-postgresql-postgis_3.4.2+dfsg-1ubuntu4_s390x.deb ... 175s Unpacking postgresql-postgis (3.4.2+dfsg-1ubuntu4) ... 175s Selecting previously unselected package python-geoalchemy2-doc. 175s Preparing to unpack .../64-python-geoalchemy2-doc_0.15.1-1_all.deb ... 175s Unpacking python-geoalchemy2-doc (0.15.1-1) ... 175s Selecting previously unselected package python3-affine. 175s Preparing to unpack .../65-python3-affine_2.4.0-1_all.deb ... 175s Unpacking python3-affine (2.4.0-1) ... 175s Selecting previously unselected package python3-mako. 175s Preparing to unpack .../66-python3-mako_1.3.5-1_all.deb ... 175s Unpacking python3-mako (1.3.5-1) ... 175s Selecting previously unselected package python3-greenlet. 175s Preparing to unpack .../67-python3-greenlet_3.0.3-0ubuntu5_s390x.deb ... 175s Unpacking python3-greenlet (3.0.3-0ubuntu5) ... 175s Selecting previously unselected package python3-sqlalchemy. 175s Preparing to unpack .../68-python3-sqlalchemy_1.4.50+ds1-1build1_all.deb ... 175s Unpacking python3-sqlalchemy (1.4.50+ds1-1build1) ... 175s Selecting previously unselected package python3-alembic. 175s Preparing to unpack .../69-python3-alembic_1.13.1-4_all.deb ... 175s Unpacking python3-alembic (1.13.1-4) ... 175s Selecting previously unselected package python3-colorama. 175s Preparing to unpack .../70-python3-colorama_0.4.6-4_all.deb ... 175s Unpacking python3-colorama (0.4.6-4) ... 175s Selecting previously unselected package python3-click. 175s Preparing to unpack .../71-python3-click_8.1.7-1_all.deb ... 175s Unpacking python3-click (8.1.7-1) ... 175s Selecting previously unselected package python3-click-plugins. 175s Preparing to unpack .../72-python3-click-plugins_1.1.1-4_all.deb ... 175s Unpacking python3-click-plugins (1.1.1-4) ... 175s Selecting previously unselected package python3-cligj. 175s Preparing to unpack .../73-python3-cligj_0.7.2-2_all.deb ... 175s Unpacking python3-cligj (0.7.2-2) ... 175s Selecting previously unselected package python3-packaging. 175s Preparing to unpack .../74-python3-packaging_24.0-1_all.deb ... 175s Unpacking python3-packaging (24.0-1) ... 175s Selecting previously unselected package python3-geoalchemy2. 175s Preparing to unpack .../75-python3-geoalchemy2_0.15.1-1_all.deb ... 175s Unpacking python3-geoalchemy2 (0.15.1-1) ... 175s Selecting previously unselected package python3-iniconfig. 175s Preparing to unpack .../76-python3-iniconfig_1.1.1-2_all.deb ... 175s Unpacking python3-iniconfig (1.1.1-2) ... 175s Selecting previously unselected package python3-mysqldb. 175s Preparing to unpack .../77-python3-mysqldb_1.4.6-2build3_s390x.deb ... 175s Unpacking python3-mysqldb (1.4.6-2build3) ... 175s Selecting previously unselected package python3-numpy. 175s Preparing to unpack .../78-python3-numpy_1%3a1.26.4+ds-10_s390x.deb ... 175s Unpacking python3-numpy (1:1.26.4+ds-10) ... 176s Selecting previously unselected package python3-pluggy. 176s Preparing to unpack .../79-python3-pluggy_1.5.0-1_all.deb ... 176s Unpacking python3-pluggy (1.5.0-1) ... 176s Selecting previously unselected package python3-psycopg2. 176s Preparing to unpack .../80-python3-psycopg2_2.9.9-1build1_s390x.deb ... 176s Unpacking python3-psycopg2 (2.9.9-1build1) ... 176s Selecting previously unselected package python3-pytest. 176s Preparing to unpack .../81-python3-pytest_7.4.4-1_all.deb ... 176s Unpacking python3-pytest (7.4.4-1) ... 176s Selecting previously unselected package python3-snuggs. 176s Preparing to unpack .../82-python3-snuggs_1.4.7-4_all.deb ... 176s Unpacking python3-snuggs (1.4.7-4) ... 176s Selecting previously unselected package python3-rasterio. 176s Preparing to unpack .../83-python3-rasterio_1.3.10-2build1_s390x.deb ... 176s Unpacking python3-rasterio (1.3.10-2build1) ... 176s Selecting previously unselected package python3-shapely. 176s Preparing to unpack .../84-python3-shapely_2.0.4-1_s390x.deb ... 176s Unpacking python3-shapely (2.0.4-1) ... 176s Selecting previously unselected package autopkgtest-satdep. 176s Preparing to unpack .../85-1-autopkgtest-satdep.deb ... 176s Unpacking autopkgtest-satdep (0) ... 176s Setting up postgresql-client-common (260) ... 176s Setting up liblcms2-2:s390x (2.14-2build1) ... 176s Setting up python3-iniconfig (1.1.1-2) ... 176s Setting up mysql-common (5.8+1.1.0build1) ... 176s update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode 176s Setting up libmysqlclient21:s390x (8.0.37-1) ... 176s Setting up proj-data (9.4.1-1) ... 176s Setting up libproj25:s390x (9.4.1-1) ... 176s Setting up libogdi4.1:s390x (4.1.1+ds-4) ... 176s Setting up python3-colorama (0.4.6-4) ... 176s Setting up libgeos3.12.2:s390x (3.12.2-1) ... 176s Setting up libpq5:s390x (16.3-1) ... 176s Setting up libqhull-r8.0:s390x (2020.2-6build1) ... 176s Setting up libxerces-c3.2t64:s390x (3.2.4+debian-1.3) ... 176s Setting up python3-click (8.1.7-1) ... 176s Setting up libaec0:s390x (1.1.3-1) ... 176s Setting up gdal-data (3.9.0+dfsg-1) ... 176s Setting up libgeotiff5:s390x (1.7.3-1) ... 176s Setting up libsnappy1v5:s390x (1.2.1-1) ... 176s Setting up python3-mysqldb (1.4.6-2build3) ... 177s Setting up libboost-serialization1.83.0:s390x (1.83.0-3ubuntu1) ... 177s Setting up libblas3:s390x (3.12.0-3build1) ... 177s update-alternatives: using /usr/lib/s390x-linux-gnu/blas/libblas.so.3 to provide /usr/lib/s390x-linux-gnu/libblas.so.3 (libblas.so.3-s390x-linux-gnu) in auto mode 177s Setting up python3-packaging (24.0-1) ... 177s Setting up libgmpxx4ldbl:s390x (2:6.3.0+dfsg-2ubuntu6) ... 177s Setting up libcfitsio10t64:s390x (4.3.1-1.1build2) ... 177s Setting up ssl-cert (1.1.2ubuntu1) ... 178s Created symlink /etc/systemd/system/multi-user.target.wants/ssl-cert.service → /usr/lib/systemd/system/ssl-cert.service. 178s Setting up python3-greenlet (3.0.3-0ubuntu5) ... 178s Setting up python3-psycopg2 (2.9.9-1build1) ... 179s Setting up python3-click-plugins (1.1.1-4) ... 179s Setting up unixodbc-common (2.3.12-1ubuntu1) ... 179s Setting up libltdl7:s390x (2.4.7-7build1) ... 179s Setting up libgfortran5:s390x (14.1.0-1ubuntu1) ... 179s Setting up python3-pluggy (1.5.0-1) ... 179s Setting up libgif7:s390x (5.2.2-1ubuntu1) ... 179s Setting up libodbc2:s390x (2.3.12-1ubuntu1) ... 179s Setting up liburiparser1:s390x (0.9.7+dfsg-2build1) ... 179s Setting up libfyba0t64:s390x (4.1.1-11build1) ... 179s Setting up libminizip1t64:s390x (1:1.3.dfsg-3.1ubuntu2) ... 179s Setting up libjson-perl (4.10000-1) ... 179s Setting up libxslt1.1:s390x (1.1.39-0exp1build1) ... 179s Setting up libblosc1:s390x (1.21.5+ds-1build1) ... 179s Setting up libopenjp2-7:s390x (2.5.0-2build3) ... 179s Setting up libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... 179s Setting up python3-affine (2.4.0-1) ... 179s Setting up libkmlbase1t64:s390x (1.3.0-12build1) ... 179s Setting up libsfcgal1t64 (1.5.1-3build1) ... 179s Setting up libsz2:s390x (1.1.3-1) ... 179s Setting up python3-mako (1.3.5-1) ... 179s Setting up libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... 179s Setting up gdal-plugins:s390x (3.9.0+dfsg-1) ... 179s Setting up postgresql-16-postgis-3-scripts (3.4.2+dfsg-1ubuntu4) ... 179s update-alternatives: using /usr/share/postgresql/16/extension/postgis-3.control to provide /usr/share/postgresql/16/extension/postgis.control (postgresql-16-postgis.control) in auto mode 179s Setting up libodbcinst2:s390x (2.3.12-1ubuntu1) ... 179s Setting up liblapack3:s390x (3.12.0-3build1) ... 179s update-alternatives: using /usr/lib/s390x-linux-gnu/lapack/liblapack.so.3 to provide /usr/lib/s390x-linux-gnu/liblapack.so.3 (liblapack.so.3-s390x-linux-gnu) in auto mode 179s Setting up libarpack2t64:s390x (3.9.1-1.1build2) ... 179s Setting up libpoppler134:s390x (24.02.0-1ubuntu10) ... 179s Setting up python3-pytest (7.4.4-1) ... 180s Setting up libgeos-c1t64:s390x (3.12.2-1) ... 180s Setting up python3-cligj (0.7.2-2) ... 180s Setting up postgresql-client-16 (16.3-1) ... 180s update-alternatives: using /usr/share/postgresql/16/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 180s Setting up libsuperlu6:s390x (6.0.1+dfsg1-1build1) ... 180s Setting up python3-sqlalchemy (1.4.50+ds1-1build1) ... 182s Setting up libkmldom1t64:s390x (1.3.0-12build1) ... 182s Setting up postgresql-common (260) ... 186s 186s Creating config file /etc/postgresql-common/createcluster.conf with new version 186s Building PostgreSQL dictionaries from installed myspell/hunspell packages... 186s Removing obsolete dictionary files: 187s Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service → /usr/lib/systemd/system/postgresql.service. 187s Setting up python3-numpy (1:1.26.4+ds-10) ... 190s Setting up libjs-sphinxdoc (7.2.6-8) ... 190s Setting up libhdf4-0-alt:s390x (4.3.0-1) ... 190s Setting up librttopo1:s390x (1.1.0-3build2) ... 190s Setting up libfreexl1:s390x (2.0.0-1build2) ... 190s Setting up python3-alembic (1.13.1-4) ... 190s Setting up postgresql-postgis-scripts (3.4.2+dfsg-1ubuntu4) ... 190s Setting up python3-geoalchemy2 (0.15.1-1) ... 191s Setting up libhdf5-103-1t64:s390x (1.10.10+repack-3.1ubuntu4) ... 191s Setting up libarmadillo12 (1:12.8.2+dfsg-1) ... 191s Setting up libspatialite8t64:s390x (5.1.0-3build1) ... 191s Setting up libhdf5-hl-100t64:s390x (1.10.10+repack-3.1ubuntu4) ... 191s Setting up python3-shapely (2.0.4-1) ... 191s Setting up libnetcdf19t64:s390x (1:4.9.2-6) ... 191s Setting up postgresql-16 (16.3-1) ... 192s Creating new PostgreSQL cluster 16/main ... 192s /usr/lib/postgresql/16/bin/initdb -D /var/lib/postgresql/16/main --auth-local peer --auth-host scram-sha-256 --no-instructions 192s The files belonging to this database system will be owned by user "postgres". 192s This user must also own the server process. 192s 192s The database cluster will be initialized with locale "C.UTF-8". 192s The default database encoding has accordingly been set to "UTF8". 192s The default text search configuration will be set to "english". 192s 192s Data page checksums are disabled. 192s 192s fixing permissions on existing directory /var/lib/postgresql/16/main ... ok 192s creating subdirectories ... ok 192s selecting dynamic shared memory implementation ... posix 192s selecting default max_connections ... 100 192s selecting default shared_buffers ... 128MB 192s selecting default time zone ... Etc/UTC 192s creating configuration files ... ok 192s running bootstrap script ... ok 199s performing post-bootstrap initialization ... ok 199s syncing data to disk ... ok 203s Setting up libkmlengine1t64:s390x (1.3.0-12build1) ... 203s Setting up python3-snuggs (1.4.7-4) ... 203s Setting up python-geoalchemy2-doc (0.15.1-1) ... 203s Setting up libgdal35:s390x (3.9.0+dfsg-1) ... 203s Setting up postgresql-16-postgis-3 (3.4.2+dfsg-1ubuntu4) ... 203s Setting up postgis (3.4.2+dfsg-1ubuntu4) ... 203s Setting up python3-rasterio (1.3.10-2build1) ... 203s Setting up postgresql-postgis (3.4.2+dfsg-1ubuntu4) ... 203s Setting up autopkgtest-satdep (0) ... 203s Processing triggers for man-db (2.12.1-2) ... 204s Processing triggers for libc-bin (2.39-0ubuntu9) ... 207s (Reading database ... 60198 files and directories currently installed.) 207s Removing autopkgtest-satdep (0) ... 208s autopkgtest [10:25:28]: test setup-db-and-run-tests.py: [----------------------- 208s The files belonging to this database system will be owned by user "ubuntu". 208s This user must also own the server process. 208s 208s The database cluster will be initialized with this locale configuration: 208s provider: libc 208s LC_COLLATE: C.UTF-8 208s LC_CTYPE: C.UTF-8 208s LC_MESSAGES: C 208s LC_MONETARY: C.UTF-8 208s LC_NUMERIC: C.UTF-8 208s LC_TIME: C.UTF-8 208s The default database encoding has accordingly been set to "UTF8". 208s The default text search configuration will be set to "english". 208s 208s Data page checksums are disabled. 208s 208s creating directory /tmp/tmpp9cx913d/data ... ok 208s creating subdirectories ... ok 208s selecting dynamic shared memory implementation ... posix 208s selecting default max_connections ... 100 208s selecting default shared_buffers ... 128MB 208s selecting default time zone ... Etc/UTC 208s creating configuration files ... ok 209s running bootstrap script ... ok 210s performing post-bootstrap initialization ... ok 216s syncing data to disk ... ok 216s 216s Success. You can now start the database server using: 216s 216s /usr/lib/postgresql/16/bin/pg_ctl -D /tmp/tmpp9cx913d/data -l logfile start 216s 221s running initdb 221s starting server 221s ============================= test session starts ============================== 221s platform linux -- Python 3.12.4, pytest-7.4.4, pluggy-1.5.0 221s rootdir: /tmp/autopkgtest.k0pZv7/build.Ryh/src 221s configfile: pyproject.toml 221s testpaths: tests 221s collected 916 items / 133 deselected / 783 selected 221s 221s tests/test_alembic_migrations.py . [ 0%] 222s tests/test_functional.py ..........s. [ 1%] 222s tests/test_functional_geopackage.py sssss [ 2%] 237s tests/test_functional_postgresql.py .......................... [ 5%] 237s tests/test_functional_sqlite.py sssssssssssssssssssss [ 8%] 238s tests/gallery/test_insert_raster.py ........... [ 9%] 238s tests/gallery/test_length_at_insert.py . [ 9%] 238s tests/gallery/test_specific_compilation.py . [ 9%] 238s tests/gallery/test_summarystatsagg.py . [ 10%] 238s tests/test_alembic_migrations.py ..s [ 10%] 242s tests/test_functional.py s..F...F...F...F...F...F...F...F...F...F...F... [ 16%] 242s F...s......ss....ssss. [ 19%] 242s tests/test_functional_sqlite.py sssssssssssssssssss [ 21%] 242s tests/test_pickle.py . [ 21%] 242s tests/gallery/test_length_at_insert.py s [ 21%] 242s tests/gallery/test_orm_mapped_v2.py s [ 22%] 242s tests/gallery/test_specific_compilation.py s [ 22%] 242s tests/test_alembic_migrations.py ss [ 22%] 242s tests/test_functional.py sssssssssssssssssssssssssssssssssssssssssssssss [ 28%] 242s ssssssssssssssssssssssss [ 31%] 242s tests/test_functional_sqlite.py sssssssssss [ 32%] 242s tests/test_pickle.py s [ 33%] 242s tests/gallery/test_length_at_insert.py s [ 33%] 242s tests/gallery/test_orm_mapped_v2.py s [ 33%] 242s tests/gallery/test_specific_compilation.py s [ 33%] 242s tests/test_alembic_migrations.py ss [ 33%] 242s tests/test_functional.py sssssssssssssssssssssssssssssssssssssssssssssss [ 39%] 242s sssssssssssssssssssss [ 42%] 242s tests/test_pickle.py s [ 42%] 242s tests/gallery/test_length_at_insert.py s [ 42%] 242s tests/gallery/test_orm_mapped_v2.py s [ 42%] 242s tests/gallery/test_specific_compilation.py s [ 42%] 242s tests/test_alembic_migrations.py ss [ 43%] 243s tests/test_functional.py sssssssssssssssssssssssssssssssssssssssssssssss [ 49%] 243s ssssssssssssss [ 50%] 243s tests/test_pickle.py s [ 51%] 243s tests/gallery/test_orm_mapped_v2.py s [ 51%] 243s tests/test_comparator.py ............................................... [ 57%] 243s ...................... [ 60%] 243s tests/test_elements.py ................................................ [ 66%] 243s tests/test_functional_geopackage.py sss [ 66%] 243s tests/test_functional_postgresql.py ..... [ 67%] 243s tests/test_functional_sqlite.py ssssssssss [ 68%] 243s tests/test_functions.py ................................................ [ 74%] 243s ........................................................................ [ 83%] 243s ........................................................................ [ 92%] 243s .......... [ 94%] 243s tests/test_shape.py ..... [ 94%] 243s tests/test_types.py ...................................... [ 99%] 243s tests/gallery/test_disable_wrapping.py .. [100%] 243s 243s =================================== FAILURES =================================== 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Point] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRI...OINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT (1.5 2.5)'}, {'geom': '0000000021e61000003ff80000000000004004000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'POINT(1.5 2.5)'}, {...651880; 0000000021e61000003ff80000000000004004000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRI...OINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT (1.5 2.5)'}, {'geom': '0000000021e61000003ff80000000000004004000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (33)! Full WKB type number was (33). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'POINT', wkt = '(1.5 2.5)' 243s use_floating_point = True 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRI...OINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT (1.5 2.5)'}, {'geom': '0000000021e61000003ff80000000000004004000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (33)! Full WKB type number was (33). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT(1.5 2.5)'}, {'geom': 'SRID=4326;POINT (1.5 2.5)'}, {'geom': '0000000021e61000003ff80000000000004004000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-LineString] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(....5 2.5, 3 4)'}, {'geom': '0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'LINESTRING(1.5 2.5,...0000000000400400000000000040080000000000004010000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(....5 2.5, 3 4)'}, {'geom': '0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (34)! Full WKB type number was (34). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'LINESTRING', wkt = '(1.5 2.5, 3 4)' 243s use_floating_point = True 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(....5 2.5, 3 4)'}, {'geom': '0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (34)! Full WKB type number was (34). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;LINESTRING (1.5 2.5, 3 4)'}, {'geom': '0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Polygon] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geo...0000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'POLYGON((1.5 2.5, 3...000000000040180000000000003ff80000000000004004000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geo...0000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (35)! Full WKB type number was (35). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'POLYGON' 243s wkt = '((1.5 2.5, 3 4, 5 6, 1.5 2.5))', use_floating_point = True 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geo...0000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (35)! Full WKB type number was (35). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': 'SRID=4326;POLYGON ((1.5 2.5, 3 4, 5 6, 1.5 2.5))'}, {'geom': '0000000023e610000000000001000000043ff8000000000000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Multi Point] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(...om': '0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'MULTIPOINT(1.5 2.5,...4004000000000000000000000140080000000000004010000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(...om': '0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (36)! Full WKB type number was (36). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'MULTIPOINT', wkt = '(1.5 2.5, 3 4)' 243s use_floating_point = True 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(...om': '0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (36)! Full WKB type number was (36). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1.5 2.5, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT ((1.5 2.5), (3 4))'}, {'geom': '0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Multi LineString] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 ...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'MULTILINESTRING((1....00000000004034000000000000403e0000000000004044000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 ...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (37)! Full WKB type number was (37). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'MULTILINESTRING' 243s wkt = '((1.5 2.5, 3 4), (10 20, 30 40))', use_floating_point = True 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 ...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (37)! Full WKB type number was (37). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING ((1.5 2.5, 3 4), (10 20, 30 40))'}, {'geom': '0000000025e6100000000000020000000002000000023ff800000000000040040000000000004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Use floating point-Multi Polygon] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPO...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'MULTIPOLYGON(((1.5 ...0000000000404e00000000000040240000000000004034000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPO...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (38)! Full WKB type number was (38). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'MULTIPOLYGON' 243s wkt = '(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))' 243s use_floating_point = True 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPO...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (38)! Full WKB type number was (38). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON (((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': '0000000026e610000000000002000000000300000001000000043ff8000000000000400400000000000040080000000000004010000000000000401400000000000040180000000000003 ... (36 characters truncated) ... 00003000000010000000440240000000000004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Point] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT (1 2)'}, {'geom': '0000000021e61000003ff00000000000004000000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'POINT(1 2)'}, {'geo...6c53d0; 0000000021e61000003ff00000000000004000000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT (1 2)'}, {'geom': '0000000021e61000003ff00000000000004000000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (33)! Full WKB type number was (33). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'POINT', wkt = '(1 2)' 243s use_floating_point = False 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT (1 2)'}, {'geom': '0000000021e61000003ff00000000000004000000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (33)! Full WKB type number was (33). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT(1 2)'}, {'geom': 'SRID=4326;POINT (1 2)'}, {'geom': '0000000021e61000003ff00000000000004000000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-LineString] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4...G (1 2, 3 4)'}, {'geom': '0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'LINESTRING(1 2, 3 4...0000000000400000000000000040080000000000004010000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4...G (1 2, 3 4)'}, {'geom': '0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (34)! Full WKB type number was (34). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'LINESTRING', wkt = '(1 2, 3 4)' 243s use_floating_point = False 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4...G (1 2, 3 4)'}, {'geom': '0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (34)! Full WKB type number was (34). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING(1 2, 3 4)'}, {'geom': 'SRID=4326;LINESTRING (1 2, 3 4)'}, {'geom': '0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Polygon] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;P...0000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'POLYGON((1 2, 3 4, ...000000000040180000000000003ff00000000000004000000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;P...0000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (35)! Full WKB type number was (35). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'POLYGON', wkt = '((1 2, 3 4, 5 6, 1 2))' 243s use_floating_point = False 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;P...0000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (35)! Full WKB type number was (35). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))'}, {'geom': 'SRID=4326;POLYGON ((1 2, 3 4, 5 6, 1 2))'}, {'geom': '0000000023e610000000000001000000043ff0000000000000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Multi Point] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4...om': '0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'MULTIPOINT(1 2, 3 4...4000000000000000000000000140080000000000004010000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4...om': '0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (36)! Full WKB type number was (36). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'MULTIPOINT', wkt = '(1 2, 3 4)' 243s use_floating_point = False 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4...om': '0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (36)! Full WKB type number was (36). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT(1 2, 3 4)'}, {'geom': 'SRID=4326;MULTIPOINT ((1 2), (3 4))'}, {'geom': '0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Multi LineString] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 4...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'MULTILINESTRING((1 ...00000000004034000000000000403e0000000000004044000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 4...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (37)! Full WKB type number was (37). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'MULTILINESTRING' 243s wkt = '((1 2, 3 4), (10 20, 30 40))', use_floating_point = False 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 4...004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (37)! Full WKB type number was (37). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))'}, {'geom': 'SRID=4326;MULTILINESTRING ((1 2, 3 4), (10 20, 30 40))'}, {'geom': '0000000025e6100000000000020000000002000000023ff000000000000040000000000000004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s _ TestInsertionCore.test_insert_all_geom_types[postgresql-Do not use floating point-Multi Polygon] _ 243s 243s self = 243s dialect = 243s constructor = > 243s statement = 'INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))' 243s parameters = ({'geom': 'MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 243s execution_options = immutabledict({'autocommit': True, 'search_path': ['gis', 'public']}) 243s args = (, [{'geom': 'MULTIPOLYGON(((1 2,...0000000000404e00000000000040240000000000004034000000000000>}], , []) 243s kw = {'cache_hit': symbol('CACHE_MISS')} 243s branched = 243s yp = None 243s conn = 243s context = 243s cursor = , evt_handled = False 243s 243s def _execute_context( 243s self, 243s dialect, 243s constructor, 243s statement, 243s parameters, 243s execution_options, 243s *args, 243s **kw 243s ): 243s """Create an :class:`.ExecutionContext` and execute, returning 243s a :class:`_engine.CursorResult`.""" 243s 243s branched = self 243s if self.__branch_from: 243s # if this is a "branched" connection, do everything in terms 243s # of the "root" connection, *except* for .close(), which is 243s # the only feature that branching provides 243s self = self.__branch_from 243s 243s if execution_options: 243s yp = execution_options.get("yield_per", None) 243s if yp: 243s execution_options = execution_options.union( 243s {"stream_results": True, "max_row_buffer": yp} 243s ) 243s 243s try: 243s conn = self._dbapi_connection 243s if conn is None: 243s conn = self._revalidate_connection() 243s 243s context = constructor( 243s dialect, self, conn, execution_options, *args, **kw 243s ) 243s except (exc.PendingRollbackError, exc.ResourceClosedError): 243s raise 243s except BaseException as e: 243s self._handle_dbapi_exception( 243s e, util.text_type(statement), parameters, None, None 243s ) 243s 243s if ( 243s self._transaction 243s and not self._transaction.is_active 243s or ( 243s self._nested_transaction 243s and not self._nested_transaction.is_active 243s ) 243s ): 243s self._invalid_transaction() 243s 243s elif self._trans_context_manager: 243s TransactionalContext._trans_ctx_check(self) 243s 243s if self._is_future and self._transaction is None: 243s self._autobegin() 243s 243s context.pre_exec() 243s 243s if dialect.use_setinputsizes: 243s context._set_input_sizes() 243s 243s cursor, statement, parameters = ( 243s context.cursor, 243s context.statement, 243s context.parameters, 243s ) 243s 243s if not context.executemany: 243s parameters = parameters[0] 243s 243s if self._has_events or self.engine._has_events: 243s for fn in self.dispatch.before_cursor_execute: 243s statement, parameters = fn( 243s self, 243s cursor, 243s statement, 243s parameters, 243s context, 243s context.executemany, 243s ) 243s 243s if self._echo: 243s 243s self._log_info(statement) 243s 243s stats = context._get_cache_stats() 243s 243s if not self.engine.hide_parameters: 243s self._log_info( 243s "[%s] %r", 243s stats, 243s sql_util._repr_params( 243s parameters, batches=10, ismulti=context.executemany 243s ), 243s ) 243s else: 243s self._log_info( 243s "[%s] [SQL parameters hidden due to hide_parameters=True]" 243s % (stats,) 243s ) 243s 243s evt_handled = False 243s try: 243s if context.executemany: 243s if self.dialect._has_events: 243s for fn in self.dialect.dispatch.do_executemany: 243s if fn(cursor, statement, parameters, context): 243s evt_handled = True 243s break 243s if not evt_handled: 243s > self.dialect.do_executemany( 243s cursor, statement, parameters, context 243s 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E psycopg2.errors.InternalError_: Unknown WKB type (38)! Full WKB type number was (38). 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError_ 243s 243s The above exception was the direct cause of the following exception: 243s 243s self = 243s dialect_name = 'postgresql', base = 243s conn = 243s metadata = MetaData(), geom_type = 'MULTIPOLYGON' 243s wkt = '(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))' 243s use_floating_point = False 243s 243s @pytest.mark.parametrize( 243s "geom_type,wkt", 243s [ 243s pytest.param("POINT", "(1 2)", id="Point"), 243s pytest.param("POINTZ", "(1 2 3)", id="Point Z"), 243s pytest.param("POINTM", "(1 2 3)", id="Point M"), 243s pytest.param("POINTZM", "(1 2 3 4)", id="Point ZM"), 243s pytest.param("LINESTRING", "(1 2, 3 4)", id="LineString"), 243s pytest.param("LINESTRINGZ", "(1 2 3, 4 5 6)", id="LineString Z"), 243s pytest.param("LINESTRINGM", "(1 2 3, 4 5 6)", id="LineString M"), 243s pytest.param("LINESTRINGZM", "(1 2 3 4, 5 6 7 8)", id="LineString ZM"), 243s pytest.param("POLYGON", "((1 2, 3 4, 5 6, 1 2))", id="Polygon"), 243s pytest.param("POLYGONZ", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon Z"), 243s pytest.param("POLYGONM", "((1 2 3, 4 5 6, 7 8 9, 1 2 3))", id="Polygon M"), 243s pytest.param( 243s "POLYGONZM", "((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))", id="Polygon ZM" 243s ), 243s pytest.param("MULTIPOINT", "(1 2, 3 4)", id="Multi Point"), 243s pytest.param("MULTIPOINTZ", "(1 2 3, 4 5 6)", id="Multi Point Z"), 243s pytest.param("MULTIPOINTM", "(1 2 3, 4 5 6)", id="Multi Point M"), 243s pytest.param("MULTIPOINTZM", "(1 2 3 4, 5 6 7 8)", id="Multi Point ZM"), 243s pytest.param("MULTILINESTRING", "((1 2, 3 4), (10 20, 30 40))", id="Multi LineString"), 243s pytest.param( 243s "MULTILINESTRINGZ", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString Z", 243s ), 243s pytest.param( 243s "MULTILINESTRINGM", 243s "((1 2 3, 4 5 6), (10 20 30, 40 50 60))", 243s id="Multi LineString M", 243s ), 243s pytest.param( 243s "MULTILINESTRINGZM", 243s "((1 2 3 4, 5 6 7 8), (10 20 30 40, 50 60 70 80))", 243s id="Multi LineString ZM", 243s ), 243s pytest.param( 243s "MULTIPOLYGON", 243s "(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))", 243s id="Multi Polygon", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZ", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon Z", 243s ), 243s pytest.param( 243s "MULTIPOLYGONM", 243s "(((1 2 3, 4 5 6, 7 8 9, 1 2 3)), ((10 20 30, 40 50 60, 70 80 90, 10 20 30)))", 243s id="Multi Polygon M", 243s ), 243s pytest.param( 243s "MULTIPOLYGONZM", 243s "(((1 2 3 4, 5 6 7 8, 9 10 11 12, 1 2 3 4))," 243s " ((10 20 30 40, 50 60 70 80, 90 100 100 120, 10 20 30 40)))", 243s id="Multi Polygon ZM", 243s ), 243s ], 243s ) 243s @pytest.mark.parametrize( 243s "use_floating_point", 243s [ 243s pytest.param(True, id="Use floating point"), 243s pytest.param(False, id="Do not use floating point"), 243s ], 243s ) 243s def test_insert_all_geom_types( 243s self, dialect_name, base, conn, metadata, geom_type, wkt, use_floating_point 243s ): 243s """Test insertion and selection of all geometry types.""" 243s ndims = 2 243s if "Z" in geom_type[-2:]: 243s ndims += 1 243s if geom_type.endswith("M"): 243s ndims += 1 243s has_m = True 243s else: 243s has_m = False 243s 243s if ndims > 2 and dialect_name == "mysql": 243s # Explicitly skip MySQL dialect to show that it can only work with 2D geometries 243s pytest.xfail(reason="MySQL only supports 2D geometry types") 243s 243s class GeomTypeTable(base): 243s __tablename__ = "test_geom_types" 243s id = Column(Integer, primary_key=True) 243s geom = Column(Geometry(srid=4326, geometry_type=geom_type, dimension=ndims)) 243s 243s metadata.drop_all(bind=conn, checkfirst=True) 243s metadata.create_all(bind=conn) 243s 243s if use_floating_point: 243s wkt = wkt.replace("1 2", "1.5 2.5") 243s 243s inserted_wkt = f"{geom_type}{wkt}" 243s 243s # Use the DB to generate the corresponding raw WKB 243s raw_wkb = conn.execute( 243s text("SELECT ST_AsBinary(ST_GeomFromText('{}', 4326))".format(inserted_wkt)) 243s ).scalar() 243s 243s wkb_elem = WKBElement(raw_wkb, srid=4326) 243s inserted_elements = [ 243s {"geom": inserted_wkt}, 243s {"geom": f"SRID=4326;{inserted_wkt}"}, 243s {"geom": WKTElement(inserted_wkt, srid=4326)}, 243s {"geom": WKTElement(f"SRID=4326;{inserted_wkt}")}, 243s ] 243s if dialect_name not in ["postgresql", "sqlite"] or not has_m: 243s # Currently Shapely does not support geometry types with M dimension 243s inserted_elements.append({"geom": wkb_elem}) 243s inserted_elements.append({"geom": wkb_elem.as_ewkb()}) 243s 243s # Insert the elements 243s > conn.execute( 243s GeomTypeTable.__table__.insert(), 243s inserted_elements, 243s ) 243s 243s tests/test_functional.py:393: 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1385: in execute 243s return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) 243s /usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py:334: in _execute_on_connection 243s return connection._execute_clauseelement( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1577: in _execute_clauseelement 243s ret = self._execute_context( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1953: in _execute_context 243s self._handle_dbapi_exception( 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:2134: in _handle_dbapi_exception 243s util.raise_( 243s /usr/lib/python3/dist-packages/sqlalchemy/util/compat.py:211: in raise_ 243s raise exception 243s /usr/lib/python3/dist-packages/sqlalchemy/engine/base.py:1890: in _execute_context 243s self.dialect.do_executemany( 243s /usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/psycopg2.py:982: in do_executemany 243s context._psycopg2_fetched_rows = xtras.execute_values( 243s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 243s 243s cur = 243s sql = b'INSERT INTO test_geom_types (geom) VALUES %s' 243s argslist = ({'geom': 'MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((...00004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'}) 243s template = '(ST_GeomFromEWKT(%(geom)s))', page_size = 1000, fetch = False 243s 243s def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False): 243s '''Execute a statement using :sql:`VALUES` with a sequence of parameters. 243s 243s :param cur: the cursor to use to execute the query. 243s 243s :param sql: the query to execute. It must contain a single ``%s`` 243s placeholder, which will be replaced by a `VALUES list`__. 243s Example: ``"INSERT INTO mytable (id, f1, f2) VALUES %s"``. 243s 243s :param argslist: sequence of sequences or dictionaries with the arguments 243s to send to the query. The type and content must be consistent with 243s *template*. 243s 243s :param template: the snippet to merge to every item in *argslist* to 243s compose the query. 243s 243s - If the *argslist* items are sequences it should contain positional 243s placeholders (e.g. ``"(%s, %s, %s)"``, or ``"(%s, %s, 42)``" if there 243s are constants value...). 243s 243s - If the *argslist* items are mappings it should contain named 243s placeholders (e.g. ``"(%(id)s, %(f1)s, 42)"``). 243s 243s If not specified, assume the arguments are sequence and use a simple 243s positional template (i.e. ``(%s, %s, ...)``), with the number of 243s placeholders sniffed by the first element in *argslist*. 243s 243s :param page_size: maximum number of *argslist* items to include in every 243s statement. If there are more items the function will execute more than 243s one statement. 243s 243s :param fetch: if `!True` return the query results into a list (like in a 243s `~cursor.fetchall()`). Useful for queries with :sql:`RETURNING` 243s clause. 243s 243s .. __: https://www.postgresql.org/docs/current/static/queries-values.html 243s 243s After the execution of the function the `cursor.rowcount` property will 243s **not** contain a total result. 243s 243s While :sql:`INSERT` is an obvious candidate for this function it is 243s possible to use it with other statements, for example:: 243s 243s >>> cur.execute( 243s ... "create table test (id int primary key, v1 int, v2 int)") 243s 243s >>> execute_values(cur, 243s ... "INSERT INTO test (id, v1, v2) VALUES %s", 243s ... [(1, 2, 3), (4, 5, 6), (7, 8, 9)]) 243s 243s >>> execute_values(cur, 243s ... """UPDATE test SET v1 = data.v1 FROM (VALUES %s) AS data (id, v1) 243s ... WHERE test.id = data.id""", 243s ... [(1, 20), (4, 50)]) 243s 243s >>> cur.execute("select * from test order by id") 243s >>> cur.fetchall() 243s [(1, 20, 3), (4, 50, 6), (7, 8, 9)]) 243s 243s ''' 243s from psycopg2.sql import Composable 243s if isinstance(sql, Composable): 243s sql = sql.as_string(cur) 243s 243s # we can't just use sql % vals because vals is bytes: if sql is bytes 243s # there will be some decoding error because of stupid codec used, and Py3 243s # doesn't implement % on bytes. 243s if not isinstance(sql, bytes): 243s sql = sql.encode(_ext.encodings[cur.connection.encoding]) 243s pre, post = _split_sql(sql) 243s 243s result = [] if fetch else None 243s for page in _paginate(argslist, page_size=page_size): 243s if template is None: 243s template = b'(' + b','.join([b'%s'] * len(page[0])) + b')' 243s parts = pre[:] 243s for args in page: 243s parts.append(cur.mogrify(template, args)) 243s parts.append(b',') 243s parts[-1:] = post 243s > cur.execute(b''.join(parts)) 243s E sqlalchemy.exc.InternalError: (psycopg2.errors.InternalError_) Unknown WKB type (38)! Full WKB type number was (38). 243s E 243s E [SQL: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT(%(geom)s))] 243s E [parameters: ({'geom': 'MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': 'SRID=4326;MULTIPOLYGON (((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))'}, {'geom': '0000000026e610000000000002000000000300000001000000043ff0000000000000400000000000000040080000000000004010000000000000401400000000000040180000000000003 ... (36 characters truncated) ... 00003000000010000000440240000000000004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000'})] 243s E (Background on this error at: https://sqlalche.me/e/14/2j85) 243s 243s /usr/lib/python3/dist-packages/psycopg2/extras.py:1299: InternalError 243s =============================== warnings summary =============================== 243s tests/__init__.py:9 243s /tmp/autopkgtest.k0pZv7/build.Ryh/src/tests/__init__.py:9: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 243s from pkg_resources import parse_version 243s 243s tests/test_functional.py::TestAdmin::test_nullable[postgresql] 243s /tmp/autopkgtest.k0pZv7/build.Ryh/src/geoalchemy2/types/__init__.py:191: UserWarning: srid not enforced when geometry_type is None 243s warnings.warn("srid not enforced when geometry_type is None") 243s 243s tests/test_functional.py::TestCallFunction::test_ST_Buffer[postgresql] 243s /tmp/autopkgtest.k0pZv7/build.Ryh/src/tests/test_functional.py:791: LegacyAPIWarning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9) 243s lake = session.query(Lake).get(lake_id) 243s 243s tests/test_functional.py::TestUpdateORM::test_other_type_fail[postgresql] 243s tests/test_pickle.py::TestPickle::test_pickle_unpickle[postgresql] 243s /tmp/autopkgtest.k0pZv7/build.Ryh/src/tests/conftest.py:200: SAWarning: transaction already deassociated from connection 243s trans.rollback() 243s 243s -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html 243s =========================== short test summary info ============================ 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Point] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-LineString] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Polygon] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Multi Point] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Multi LineString] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Use floating point-Multi Polygon] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Point] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-LineString] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Polygon] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Multi Point] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Multi LineString] 243s FAILED tests/test_functional.py::TestInsertionCore::test_insert_all_geom_types[postgresql-Do not use floating point-Multi Polygon] 243s === 12 failed, 473 passed, 298 skipped, 133 deselected, 5 warnings in 23.34s === 243s stopping postgres server 243s postgres server error log 243s 243s 2024-06-16 10:25:36.108 UTC [4964] LOG: starting PostgreSQL 16.3 (Ubuntu 16.3-1) on s390x-ibm-linux-gnu, compiled by gcc (Ubuntu 13.2.0-25ubuntu1) 13.2.0, 64-bit 243s 2024-06-16 10:25:36.108 UTC [4964] LOG: listening on IPv6 address "::1", port 36933 243s 2024-06-16 10:25:36.108 UTC [4964] LOG: listening on IPv4 address "127.0.0.1", port 36933 243s 2024-06-16 10:25:36.108 UTC [4964] LOG: listening on Unix socket "/tmp/tmpp9cx913d/tmp/.s.PGSQL.36933" 243s 2024-06-16 10:25:36.111 UTC [4967] LOG: database system was shut down at 2024-06-16 10:25:30 UTC 243s 2024-06-16 10:25:36.116 UTC [4964] LOG: database system is ready to accept connections 243s 2024-06-16 10:25:41.605 UTC [4973] WARNING: there is no transaction in progress 243s 2024-06-16 10:25:41.704 UTC [4984] ERROR: null value in column "geom_not_nullable" of relation "nullable_geom_type" violates not-null constraint 243s 2024-06-16 10:25:41.704 UTC [4984] DETAIL: Failing row contains (3, null, null, 0020000002000010E600000002000000000000000000000000000000003FF000..., null). 243s 2024-06-16 10:25:41.704 UTC [4984] STATEMENT: INSERT INTO nullable_geom_type (geom_not_nullable, geom_nullable, geom_col_not_nullable, geom_col_nullable) VALUES (ST_GeomFromEWKT(NULL), ST_GeomFromEWKT(NULL), ST_GeomFromEWKT('SRID=4326;LINESTRING(0 0,1 1)'), ST_GeomFromEWKT(NULL)) RETURNING nullable_geom_type.id 243s 2024-06-16 10:25:41.705 UTC [4984] ERROR: null value in column "geom_col_not_nullable" of relation "nullable_geom_type" violates not-null constraint 243s 2024-06-16 10:25:41.705 UTC [4984] DETAIL: Failing row contains (4, 0020000002000010E600000002000000000000000000000000000000003FF000..., null, null, null). 243s 2024-06-16 10:25:41.705 UTC [4984] STATEMENT: INSERT INTO nullable_geom_type (geom_not_nullable, geom_nullable, geom_col_not_nullable, geom_col_nullable) VALUES (ST_GeomFromEWKT('SRID=4326;LINESTRING(0 0,1 1)'), ST_GeomFromEWKT(NULL), ST_GeomFromEWKT(NULL), ST_GeomFromEWKT(NULL)) RETURNING nullable_geom_type.id 243s 2024-06-16 10:25:41.892 UTC [4991] ERROR: new row for relation "contrained_lake" violates check constraint "check_geom_sk" 243s 2024-06-16 10:25:41.892 UTC [4991] DETAIL: Failing row contains (4, null, should fail, null). 243s 2024-06-16 10:25:41.892 UTC [4991] STATEMENT: INSERT INTO contrained_lake (a_str, checked_str, geom) VALUES (NULL, 'should fail', ST_GeomFromEWKT(NULL)) RETURNING contrained_lake.id 243s 2024-06-16 10:25:57.055 UTC [5003] ERROR: contains: Operation on mixed SRID geometries (Polygon, 4326) != (Point, 0) 243s 2024-06-16 10:25:57.055 UTC [5003] STATEMENT: SELECT gis.lake.id AS gis_lake_id, ST_AsEWKB(gis.lake.geom) AS gis_lake_geom 243s FROM gis.lake 243s WHERE ST_Within('POINT(0 0)', ST_Buffer(gis.lake.geom, 2)) 243s 2024-06-16 10:25:58.753 UTC [5039] ERROR: Unknown WKB type (33)! Full WKB type number was (33). 243s 2024-06-16 10:25:58.753 UTC [5039] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('POINT(1.5 2.5)')),(ST_GeomFromEWKT('SRID=4326;POINT(1.5 2.5)')),(ST_GeomFromEWKT('SRID=4326;POINT(1.5 2.5)')),(ST_GeomFromEWKT('SRID=4326;POINT(1.5 2.5)')),(ST_GeomFromEWKT('SRID=4326;POINT (1.5 2.5)')),(ST_GeomFromEWKT('0000000021e61000003ff80000000000004004000000000000')) 243s 2024-06-16 10:25:59.057 UTC [5043] ERROR: Unknown WKB type (34)! Full WKB type number was (34). 243s 2024-06-16 10:25:59.057 UTC [5043] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('LINESTRING(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING (1.5 2.5, 3 4)')),(ST_GeomFromEWKT('0000000022e6100000000000023ff8000000000000400400000000000040080000000000004010000000000000')) 243s 2024-06-16 10:25:59.312 UTC [5047] ERROR: Unknown WKB type (35)! Full WKB type number was (35). 243s 2024-06-16 10:25:59.312 UTC [5047] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('SRID=4326;POLYGON ((1.5 2.5, 3 4, 5 6, 1.5 2.5))')),(ST_GeomFromEWKT('0000000023e610000000000001000000043ff8000000000000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff80000000000004004000000000000')) 243s 2024-06-16 10:25:59.572 UTC [5051] ERROR: Unknown WKB type (36)! Full WKB type number was (36). 243s 2024-06-16 10:25:59.572 UTC [5051] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTIPOINT(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1.5 2.5, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT ((1.5 2.5), (3 4))')),(ST_GeomFromEWKT('0000000024e61000000000000200000000013ff80000000000004004000000000000000000000140080000000000004010000000000000')) 243s 2024-06-16 10:25:59.845 UTC [5055] ERROR: Unknown WKB type (37)! Full WKB type number was (37). 243s 2024-06-16 10:25:59.845 UTC [5055] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING ((1.5 2.5, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('0000000025e6100000000000020000000002000000023ff800000000000040040000000000004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000')) 243s 2024-06-16 10:26:00.190 UTC [5059] ERROR: Unknown WKB type (38)! Full WKB type number was (38). 243s 2024-06-16 10:26:00.190 UTC [5059] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON (((1.5 2.5, 3 4, 5 6, 1.5 2.5)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('0000000026e610000000000002000000000300000001000000043ff8000000000000400400000000000040080000000000004010000000000000401400000000000040180000000000003ff800000000000040040000000000000000000003000000010000000440240000000000004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000')) 243s 2024-06-16 10:26:00.475 UTC [5063] ERROR: Unknown WKB type (33)! Full WKB type number was (33). 243s 2024-06-16 10:26:00.475 UTC [5063] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('POINT(1 2)')),(ST_GeomFromEWKT('SRID=4326;POINT(1 2)')),(ST_GeomFromEWKT('SRID=4326;POINT(1 2)')),(ST_GeomFromEWKT('SRID=4326;POINT(1 2)')),(ST_GeomFromEWKT('SRID=4326;POINT (1 2)')),(ST_GeomFromEWKT('0000000021e61000003ff00000000000004000000000000000')) 243s 2024-06-16 10:26:00.752 UTC [5067] ERROR: Unknown WKB type (34)! Full WKB type number was (34). 243s 2024-06-16 10:26:00.752 UTC [5067] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('LINESTRING(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;LINESTRING (1 2, 3 4)')),(ST_GeomFromEWKT('0000000022e6100000000000023ff0000000000000400000000000000040080000000000004010000000000000')) 243s 2024-06-16 10:26:01.081 UTC [5071] ERROR: Unknown WKB type (35)! Full WKB type number was (35). 243s 2024-06-16 10:26:01.081 UTC [5071] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('POLYGON((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('SRID=4326;POLYGON((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('SRID=4326;POLYGON ((1 2, 3 4, 5 6, 1 2))')),(ST_GeomFromEWKT('0000000023e610000000000001000000043ff0000000000000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff00000000000004000000000000000')) 243s 2024-06-16 10:26:01.453 UTC [5075] ERROR: Unknown WKB type (36)! Full WKB type number was (36). 243s 2024-06-16 10:26:01.453 UTC [5075] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTIPOINT(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT(1 2, 3 4)')),(ST_GeomFromEWKT('SRID=4326;MULTIPOINT ((1 2), (3 4))')),(ST_GeomFromEWKT('0000000024e61000000000000200000000013ff00000000000004000000000000000000000000140080000000000004010000000000000')) 243s 2024-06-16 10:26:01.731 UTC [5079] ERROR: Unknown WKB type (37)! Full WKB type number was (37). 243s 2024-06-16 10:26:01.731 UTC [5079] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTILINESTRING((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('SRID=4326;MULTILINESTRING ((1 2, 3 4), (10 20, 30 40))')),(ST_GeomFromEWKT('0000000025e6100000000000020000000002000000023ff000000000000040000000000000004008000000000000401000000000000000000000020000000240240000000000004034000000000000403e0000000000004044000000000000')) 243s 2024-06-16 10:26:02.025 UTC [5083] ERROR: Unknown WKB type (38)! Full WKB type number was (38). 243s 2024-06-16 10:26:02.025 UTC [5083] STATEMENT: INSERT INTO test_geom_types (geom) VALUES (ST_GeomFromEWKT('MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON(((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('SRID=4326;MULTIPOLYGON (((1 2, 3 4, 5 6, 1 2)), ((10 20, 30 40, 50 60, 10 20)))')),(ST_GeomFromEWKT('0000000026e610000000000002000000000300000001000000043ff0000000000000400000000000000040080000000000004010000000000000401400000000000040180000000000003ff000000000000040000000000000000000000003000000010000000440240000000000004034000000000000403e00000000000040440000000000004049000000000000404e00000000000040240000000000004034000000000000')) 243s 2024-06-16 10:26:02.372 UTC [5091] ERROR: function st_geomfromewkt(integer) does not exist at character 26 243s 2024-06-16 10:26:02.372 UTC [5091] HINT: No function matches the given name and argument types. You might need to add explicit type casts. 243s 2024-06-16 10:26:02.372 UTC [5091] STATEMENT: UPDATE gis.lake SET geom=ST_GeomFromEWKT(1) WHERE gis.lake.id = 1 243s 2024-06-16 10:26:02.420 UTC [5093] ERROR: function st_unknownfunction(geometry, integer) does not exist at character 8 243s 2024-06-16 10:26:02.420 UTC [5093] HINT: No function matches the given name and argument types. You might need to add explicit type casts. 243s 2024-06-16 10:26:02.420 UTC [5093] STATEMENT: SELECT ST_UnknownFunction(gis.lake.geom, 2) AS "ST_UnknownFunction_1" 243s FROM gis.lake 243s 2024-06-16 10:26:03.774 UTC [4964] LOG: received smart shutdown request 243s 2024-06-16 10:26:03.780 UTC [4964] LOG: background worker "logical replication launcher" (PID 4970) exited with exit code 1 243s 2024-06-16 10:26:03.782 UTC [4965] LOG: shutting down 243s 2024-06-16 10:26:03.782 UTC [4965] LOG: checkpoint starting: shutdown immediate 243s 2024-06-16 10:26:03.810 UTC [4965] LOG: checkpoint complete: wrote 2143 buffers (13.1%); 0 WAL file(s) added, 0 removed, 1 recycled; write=0.016 s, sync=0.001 s, total=0.028 s; sync files=0, longest=0.000 s, average=0.000 s; distance=17581 kB, estimate=17581 kB; lsn=0/261A478, redo lsn=0/261A478 243s 2024-06-16 10:26:03.821 UTC [4964] LOG: database system is shut down 243s 243s clean up 244s autopkgtest [10:26:04]: test setup-db-and-run-tests.py: -----------------------] 245s autopkgtest [10:26:05]: test setup-db-and-run-tests.py: - - - - - - - - - - results - - - - - - - - - - 245s setup-db-and-run-tests.py FAIL non-zero exit status 1 245s autopkgtest [10:26:05]: @@@@@@@@@@@@@@@@@@@@ summary 245s setup-db-and-run-tests.py FAIL non-zero exit status 1 258s nova [W] Using flock in scalingstack-bos01-s390x 258s flock: timeout while waiting to get lock 258s Creating nova instance adt-oracular-s390x-geoalchemy2-20240616-102200-juju-7f2275-prod-proposed-migration-environment-3-fb87bd89-0a0e-4544-95a3-8244b7133df3 from image adt/ubuntu-oracular-s390x-server-20240616.img (UUID d6f70b60-e0c1-480d-8231-cedbbc2f917e)...