Friday 11 December 2020

Re: Ubuntu Focal update of broken Calibre package

Hello,

I recently wanted to test out Calibre with a newer Kindle Fire tablet. It turns out that I ran into many of the issues mentioned with the broken 4.99.4+dfsg+really4.12.0-1build1 package version.  I was able to get this old version of Calibre working to identify the tablet on Focal with an updated version of libmtp.  This is in my PPA here: https://launchpad.net/~trinitronx/+archive/ubuntu/calibre

So, knowing some packaging tricks for Ubuntu & Debian, I decided to try and create an updated Calibre package in a PPA for Focal / 20.04.1.  I ran into the issue of first needing to backport all the build dependencies.  I started out by using the same version of Calibre ( 5.6.0+dfsg ) from the source package in Ubuntu groovy, and then after reading this thread trying to backport the versions from Debian bullseye & sid.  So far I've been able to backport the following:

  • pyqt-builder  1.4.0+dfsg-1 and 1.6.0+dfsg-1
  • sip4  4.19.24+dfsg-1
  • sip5  5.3.0+dfsg-2  and 5.5.0+dfsg-1
  • pyqt5-sip 12.8.1-1
Note: I also backported debhelper 13.2 in that PPA, but ended up not using it because strict build-depends on "debhelper (=12)" is on most of the other packages in Focal, and Soyuz / pbuilder will fail on that build dependency.  Instead, I just switched back to "debhelper (=12)" in the source package's debian/control files.
 
Most packages I mentioned will need a rebuild and a patch to use a different  location for PyQt5 *.sip files.  

This explains exactly the issues I ran into for the main Calibre package! ðŸ˜Š
So far I have tried two routes to resolve the build dependency:

  • sip4 4.19.24+dfsg-1 + pyqt-builder 1.4.0+dfsg-1
    • Patched Calibre's setup.py to use Python3
    • Patched Calibre's setup/build.py to use sip-module = "sip" 
    • Ran into issues with python3-sipbuilder
      • It could not find the package path, as you described:
        SIPing 3 files...
        /usr/bin/python3.8 -c import os; os.chdir('/media/terabyte/src/pub/calibre-debian/calibre/build/pyqt/pictureflow'); from sipbuild.tools.build import main; main(); --verbose --no-make --qmake /usr/bin/qmake
        Querying qmake about your Qt installation...
        /usr/bin/qmake -query
        These bindings will be built: pictureflow.
        Generating the pictureflow bindings...
        -c: Unable to find file "QtWidgets/QtWidgetsmod.sip"

      • There was a problem with how python3-sipbuilder's "get_package_dir" function was working.
        The "sip_parts" result of the split only had 1 array item: ["sip"]
        The code discards the last array item, assuming it's a filename extension.
        Therefore, it creates an empty array that gets passed to: os.path.join()
  • sip5 5.3.0+dfsg-2  + pyqt-builder 1.4.0+dfsg-1 + pyqt5-dev 5.14.1+dfsg-3build1
    • This option required just patching Calibre's setup.py to use Python3
    • Tried downgrading Build-Depends to pyqt5-dev (>= 5.14.1+dfsg-3build1)
    • I became blocked at first on backporting the build dependencies for the pyqt5-sip / PyQt5.sip and many other pyqt5 packages.  Not sure what the issue was (I've never seen or used PyQt5 or SIP before this... seemed like a large rabbit hole)
  •  sip5 5.5.0+dfsg-1 + pyqt-builder / python3-pyqtbuild 1.6.0+dfsg-1
     + pyqt5-sip / python3-pyqt5.sip
     12.8.1-1 + pyqt5 
    5.15.2+dfsg-1 (all backported)
    • Again, just added the patch Calibre's setup.py to use Python3
    • This time it's missing a package: python3-pyqt5.qtwebengine

      dpkg-checkbuilddeps: error: Unmet build dependencies: python3-pyqt5.qtwebengine (>= 5.12.1-4+b1)
       

You can see this in Debian: after I uploaded new PyQt5 (built with SIP 5) to  unstable, immediately some packages started to fail to build from source:    - https://bugs.debian.org/971173 (krita)  - https://bugs.debian.org/971217 (python-poppler-qt5)  - https://bugs.debian.org/971172 (veusz)    And packages that were not rebuilt got runtime errors:    - https://bugs.debian.org/971538 (qgis)  - https://bugs.debian.org/970921 (calibre)    So what I said is true: this requires simultaneous changes in many packages,  even if it's just a rebuild for some of them.


I think if I or someone else here can figure out how to get python3-pyqt5.qtwebengine to build, then Calibre may also be able to build with this set of dependencies.  Note, I'm not an "official" Debian or Ubuntu developer and have not gone through the onboarding / mentorship process to become one.  So for now, these efforts are for a Focal targeted PPA only.  However, with some help this could be the first step towards getting an official set into focal-backports.

I would appreciate any help with figuring out how to get that last python3-pyqt5.qtwebengine package to be generated from the pyqt5 source package.  I'm assuming this is where it comes from, given that all the other similarly named ones were generated from that one.  Also, the Debian package site shows this as the source package: https://packages.debian.org/stretch/python3-pyqt5.qtwebengine

Thanks,
- James Cuzella