Hi all,
I did my own first +1 shift last week (Virtual RM Engineering Sprint Week, so people were very busy). After synchronization with Benjamin Drung for the week, I ran find-proposed-cluster form ubuntu-archive-tools:
❯ ./find-proposed-cluster
rust-gix 17
linux-restricted-modules 5
rust-sequoia-keystore 4
rust-glycin 3
rust-cargo 3
So I focused in rust-gix... spoiler: No luck. The suspect seemed to me to be rust-hashbrown. I followed the "Analyzing migrations: britney, update_output.txt, chdist, dose-distcheck, apt solver 3" mail from Adrien Nader in ubuntu-devel-discuss (Thanks Adrien!) but locally all was installable, no erros/warning messages. rust-memosffset was all installable too... Sadly, I had no clue about was happening with Britney.
So then I went trough the update excuses, triggering a lot of 'unexpected eof from the testbed' for armhf and s390x mainly.
Finally, I worked on the following packages, with different outcome:
***** Successfull fixes:
- mako & sphinx: mako was FTBFS with the following error
dh_sphinxdoc: error: debian/python-mako-doc/usr/share/doc/python-mako-doc/html/search.html does not load searchindex.js
Researching it, the culprit effectively was sphinx. I opened a LP bug [1] an submitted a MR to Salsa [2] (I found also a open bug in Debian [3]). The fix was accepted and synced in Ubuntu, after taking care of re-triggering some tests (how many sphinx has!) in the migration. Mako also migratted since then.
- fftw: FTBFS with new gcc-14. There was a bug opened in Debian already [4], and I sent a patch fixing the issue:
fftw_f77_mpi.h:50:48: error: passing argument 1 of 'MPI_Comm_f2c' makes integer from pointer without a cast [-Wint-conversion]
GCC no longer treats integer types and pointer types as equivalent in assignments (including implied assignments of function
arguments + and return values), and instead fails the compilation with a type error. The MPI_Comm_f2c function from MPICH receives
a MPI_Fint comm [5], but here this was done in an implicit way with the accesed value. The patch casts it to the recommended
standard types intptr_t.
The package was synced in Ubuntu later with the fix (in some way).
- form: FTBFS with new gcc-14. Erich Eickmeyer had already opened a bug for it [6]. I went through all the no-longer-applies implicit casting making then explicit to the expected type, and I created a patch that Erich reviewed and sponsored (Thanks Erich!) [7]. I submitted the patch to Debian opening a bug there [7.1]
******* Still pending for review & uploading.
- glc27: This is one of the affected by dpkg ELF package metadata missing, tracked by Benjamin Drung at [8], so I added it to the list and proposed a fix here [9].
******* Incomplete (needs further investigation):
-pystac: This is pending on the new version of python3-jsonschema, which presents a component mismatches issue (known as it's being handled). I opened bug 2075152 [10] to reflect this, adding also that dt-schema is blocking the migration with failing tests. so...
-dt-schema: This package doesn't work with versions of jsonschema greater or equal than 4.18. I found some upstream's issues referencing this [11][12], but also the I found following commit [13] in some dt-schemas 2924 versions to move to the old way while they don't support new jsonschema. I applied as a patch but it didn't work.
Later I saw Benjaming open a bug for removal at [14].
-pcp: A FTBFS in Ubuntu (except armhf and s390x) but not in Debian. It seems something with the flags is happening. I did some attempts and finally open a LP bug 2076092 [15].
-magicgui: This is blocking pysignal. I found a bug opened in Debian [16]. In upstream all it's passing for python 3.12 [17], so I was looking for particular commits that could fix them (e.g. [18], but it didn't work) and mimic also upstream github actions backend with qt6. I opened a LP bug 2076095 [19].
- form: When fixing the FTBFS, I realized it has a ruby-based testsuite disabled. I opened a LP bug 2075722 for this [20].
- dask: It's preventing pandas. The log [21] reports things like:
1109s ==================================== ERRORS ====================================
1109s _________________________ ERROR at setup of test_empty _________________________
1109s
1109s @pytest.fixture
1109s def db():
1109s with tmpfile() as f:
1109s uri = "sqlite:///%s" % f
1109s > df.to_sql("test", uri, index=True, if_exists="replace")
1109s
1109s /usr/lib/python3/dist-packages/dask/dataframe/io/tests/test_sql.py:39:
1109s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
1109s /usr/lib/python3/dist-packages/pandas/util/_decorators.py:333: in wrapper
1109s return func(*args, **kwargs)
1109s /usr/lib/python3/dist-packages/pandas/core/generic.py:3087: in to_sql
1109s return sql.to_sql(
1109s /usr/lib/python3/dist-packages/pandas/io/sql.py:841: in to_sql
1109s with pandasSQL_builder(con, schema=schema, need_transaction=True) as pandas_sql:
1109s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
1109s
1109s con = 'sqlite:////tmp/tmphmlgfo8g', schema = None, need_transaction = True
1109s
1109s def pandasSQL_builder(
1109s con,
1109s schema: str | None = None,
1109s need_transaction: bool = False,
1109s ) -> PandasSQL:
1109s """
1109s Convenience function to return the correct PandasSQL subclass based on the
1109s provided parameters. Also creates a sqlalchemy connection and transaction
1109s if necessary.
1109s """
1109s import sqlite3
1109s
1109s if isinstance(con, sqlite3.Connection) or con is None:
1109s return SQLiteDatabase(con)
1109s
1109s sqlalchemy = import_optional_dependency("sqlalchemy", errors="ignore")
1109s
1109s if isinstance(con, str) and sqlalchemy is None:
1109s > raise ImportError("Using URI string without sqlalchemy installed.")
1109s E ImportError: Using URI string without sqlalchemy installed.
but python3-sqlalchemy is in the control file... Digging more, last pandas has a change in the version of sqlachemy that uses, as we can see in the diff [22] that even debian remove the depency:
- python3-sqlalchemy (>= 1.4.36~) [!ia64 !hppa !sh4 !x32] <!nocheck> <!nodoc>,
+# we don't have this version python3-sqlalchemy (>= 2.0.0~) [!ia64 !hppa !sh4 !x32] <!nocheck> <!nodoc>,
For us, sqlalchemy is also migrating for 35 days [23].
That's all. Thanks also to Simon Chopin, Jeremy Bicha, Benjamin Drung and Erich Eickmeyer for their comments and support.
See you (after my holiday period :) )!
[1] https://bugs.launchpad.net/sphinx/+bug/2074525
[2] https://salsa.debian.org/python-team/packages/sphinx/-/merge_requests/5
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1076969
[4] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1074955
[5] https://www.mpich.org/static/docs/latest/www3/MPI_Comm_f2c.html
[6] https://launchpad.net/bugs/2075550
[7] https://code.launchpad.net/~mirespace/ubuntu/+source/form/+git/form/+merge/470565
[7.1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1077815
[8] https://bugs.launchpad.net/ubuntu/+source/dpkg/+bug/2071468
[9] https://code.launchpad.net/~mirespace/ubuntu/+source/gcl27/+git/gcl27/+merge/470568
[10] https://bugs.launchpad.net/ubuntu/+source/pystac/+bug/2075152
[11] https://github.com/devicetree-org/dt-schema/issues/108
[12] https://github.com/devicetree-org/dt-schema/issues/109
[13] https://github.com/devicetree-org/dt-schema/commit/fb80ec43c2044d32cf576ef7c660eedf9c38f9ae
[14] https://bugs.launchpad.net/ubuntu/+source/dt-schema/+bug/2075957
[15] https://bugs.launchpad.net/ubuntu/+source/pcp/+bug/2076092
[16] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1074724
[17] https://github.com/pyapp-kit/magicgui/actions/runs/10240968558/job/28328574711
[18] https://github.com/pyapp-kit/magicgui/pull/630
[19] https://bugs.launchpad.net/ubuntu/+source/magicgui/+bug/2076095
[20] https://bugs.launchpad.net/ubuntu/+source/form/+bug/2075722
[21] https://objectstorage.prodstack5.canonical.com/swift/v1/AUTH_0f9aae918d5b4744bf7b827671c86842/autopkgtest-oracular/oracular/amd64/d/dask/20240804_074029_01e52@/log.gz
[22] https://launchpadlibrarian.net/738462708/pandas_2.1.4+dfsg-8_2.2.2+dfsg-4.diff.gz
[23] https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.html#sqlalchemy
--
I did my own first +1 shift last week (Virtual RM Engineering Sprint Week, so people were very busy). After synchronization with Benjamin Drung for the week, I ran find-proposed-cluster form ubuntu-archive-tools:
❯ ./find-proposed-cluster
rust-gix 17
linux-restricted-modules 5
rust-sequoia-keystore 4
rust-glycin 3
rust-cargo 3
So I focused in rust-gix... spoiler: No luck. The suspect seemed to me to be rust-hashbrown. I followed the "Analyzing migrations: britney, update_output.txt, chdist, dose-distcheck, apt solver 3" mail from Adrien Nader in ubuntu-devel-discuss (Thanks Adrien!) but locally all was installable, no erros/warning messages. rust-memosffset was all installable too... Sadly, I had no clue about was happening with Britney.
So then I went trough the update excuses, triggering a lot of 'unexpected eof from the testbed' for armhf and s390x mainly.
Finally, I worked on the following packages, with different outcome:
***** Successfull fixes:
- mako & sphinx: mako was FTBFS with the following error
dh_sphinxdoc: error: debian/python-mako-doc/usr/share/doc/python-mako-doc/html/search.html does not load searchindex.js
Researching it, the culprit effectively was sphinx. I opened a LP bug [1] an submitted a MR to Salsa [2] (I found also a open bug in Debian [3]). The fix was accepted and synced in Ubuntu, after taking care of re-triggering some tests (how many sphinx has!) in the migration. Mako also migratted since then.
- fftw: FTBFS with new gcc-14. There was a bug opened in Debian already [4], and I sent a patch fixing the issue:
fftw_f77_mpi.h:50:48: error: passing argument 1 of 'MPI_Comm_f2c' makes integer from pointer without a cast [-Wint-conversion]
GCC no longer treats integer types and pointer types as equivalent in assignments (including implied assignments of function
arguments + and return values), and instead fails the compilation with a type error. The MPI_Comm_f2c function from MPICH receives
a MPI_Fint comm [5], but here this was done in an implicit way with the accesed value. The patch casts it to the recommended
standard types intptr_t.
The package was synced in Ubuntu later with the fix (in some way).
- form: FTBFS with new gcc-14. Erich Eickmeyer had already opened a bug for it [6]. I went through all the no-longer-applies implicit casting making then explicit to the expected type, and I created a patch that Erich reviewed and sponsored (Thanks Erich!) [7]. I submitted the patch to Debian opening a bug there [7.1]
******* Still pending for review & uploading.
- glc27: This is one of the affected by dpkg ELF package metadata missing, tracked by Benjamin Drung at [8], so I added it to the list and proposed a fix here [9].
******* Incomplete (needs further investigation):
-pystac: This is pending on the new version of python3-jsonschema, which presents a component mismatches issue (known as it's being handled). I opened bug 2075152 [10] to reflect this, adding also that dt-schema is blocking the migration with failing tests. so...
-dt-schema: This package doesn't work with versions of jsonschema greater or equal than 4.18. I found some upstream's issues referencing this [11][12], but also the I found following commit [13] in some dt-schemas 2924 versions to move to the old way while they don't support new jsonschema. I applied as a patch but it didn't work.
Later I saw Benjaming open a bug for removal at [14].
-pcp: A FTBFS in Ubuntu (except armhf and s390x) but not in Debian. It seems something with the flags is happening. I did some attempts and finally open a LP bug 2076092 [15].
-magicgui: This is blocking pysignal. I found a bug opened in Debian [16]. In upstream all it's passing for python 3.12 [17], so I was looking for particular commits that could fix them (e.g. [18], but it didn't work) and mimic also upstream github actions backend with qt6. I opened a LP bug 2076095 [19].
- form: When fixing the FTBFS, I realized it has a ruby-based testsuite disabled. I opened a LP bug 2075722 for this [20].
- dask: It's preventing pandas. The log [21] reports things like:
1109s ==================================== ERRORS ====================================
1109s _________________________ ERROR at setup of test_empty _________________________
1109s
1109s @pytest.fixture
1109s def db():
1109s with tmpfile() as f:
1109s uri = "sqlite:///%s" % f
1109s > df.to_sql("test", uri, index=True, if_exists="replace")
1109s
1109s /usr/lib/python3/dist-packages/dask/dataframe/io/tests/test_sql.py:39:
1109s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
1109s /usr/lib/python3/dist-packages/pandas/util/_decorators.py:333: in wrapper
1109s return func(*args, **kwargs)
1109s /usr/lib/python3/dist-packages/pandas/core/generic.py:3087: in to_sql
1109s return sql.to_sql(
1109s /usr/lib/python3/dist-packages/pandas/io/sql.py:841: in to_sql
1109s with pandasSQL_builder(con, schema=schema, need_transaction=True) as pandas_sql:
1109s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
1109s
1109s con = 'sqlite:////tmp/tmphmlgfo8g', schema = None, need_transaction = True
1109s
1109s def pandasSQL_builder(
1109s con,
1109s schema: str | None = None,
1109s need_transaction: bool = False,
1109s ) -> PandasSQL:
1109s """
1109s Convenience function to return the correct PandasSQL subclass based on the
1109s provided parameters. Also creates a sqlalchemy connection and transaction
1109s if necessary.
1109s """
1109s import sqlite3
1109s
1109s if isinstance(con, sqlite3.Connection) or con is None:
1109s return SQLiteDatabase(con)
1109s
1109s sqlalchemy = import_optional_dependency("sqlalchemy", errors="ignore")
1109s
1109s if isinstance(con, str) and sqlalchemy is None:
1109s > raise ImportError("Using URI string without sqlalchemy installed.")
1109s E ImportError: Using URI string without sqlalchemy installed.
but python3-sqlalchemy is in the control file... Digging more, last pandas has a change in the version of sqlachemy that uses, as we can see in the diff [22] that even debian remove the depency:
- python3-sqlalchemy (>= 1.4.36~) [!ia64 !hppa !sh4 !x32] <!nocheck> <!nodoc>,
+# we don't have this version python3-sqlalchemy (>= 2.0.0~) [!ia64 !hppa !sh4 !x32] <!nocheck> <!nodoc>,
For us, sqlalchemy is also migrating for 35 days [23].
That's all. Thanks also to Simon Chopin, Jeremy Bicha, Benjamin Drung and Erich Eickmeyer for their comments and support.
See you (after my holiday period :) )!
[1] https://bugs.launchpad.net/sphinx/+bug/2074525
[2] https://salsa.debian.org/python-team/packages/sphinx/-/merge_requests/5
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1076969
[4] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1074955
[5] https://www.mpich.org/static/docs/latest/www3/MPI_Comm_f2c.html
[6] https://launchpad.net/bugs/2075550
[7] https://code.launchpad.net/~mirespace/ubuntu/+source/form/+git/form/+merge/470565
[7.1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1077815
[8] https://bugs.launchpad.net/ubuntu/+source/dpkg/+bug/2071468
[9] https://code.launchpad.net/~mirespace/ubuntu/+source/gcl27/+git/gcl27/+merge/470568
[10] https://bugs.launchpad.net/ubuntu/+source/pystac/+bug/2075152
[11] https://github.com/devicetree-org/dt-schema/issues/108
[12] https://github.com/devicetree-org/dt-schema/issues/109
[13] https://github.com/devicetree-org/dt-schema/commit/fb80ec43c2044d32cf576ef7c660eedf9c38f9ae
[14] https://bugs.launchpad.net/ubuntu/+source/dt-schema/+bug/2075957
[15] https://bugs.launchpad.net/ubuntu/+source/pcp/+bug/2076092
[16] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1074724
[17] https://github.com/pyapp-kit/magicgui/actions/runs/10240968558/job/28328574711
[18] https://github.com/pyapp-kit/magicgui/pull/630
[19] https://bugs.launchpad.net/ubuntu/+source/magicgui/+bug/2076095
[20] https://bugs.launchpad.net/ubuntu/+source/form/+bug/2075722
[21] https://objectstorage.prodstack5.canonical.com/swift/v1/AUTH_0f9aae918d5b4744bf7b827671c86842/autopkgtest-oracular/oracular/amd64/d/dask/20240804_074029_01e52@/log.gz
[22] https://launchpadlibrarian.net/738462708/pandas_2.1.4+dfsg-8_2.2.2+dfsg-4.diff.gz
[23] https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.html#sqlalchemy
 | |
|---|---|
Miriam España Acebal | |
Software Engineer II - Ubuntu Public Cloud/Server | |
Email: | |
Location: | Spain (GMT+2) |
canonical.com | |
ubuntu.com | |
