Thursday, 25 September 2025

Re: PGP key recommendations for Ubuntu Development

On Wed, Sep 24, 2025 at 4:10 PM Aaron Rainbolt <arraybolt3@gmail.com> wrote:
>
> On Wed, Sep 24, 2025 at 4:53 AM Christian Ehrhardt
> <christian.ehrhardt@canonical.com> wrote:
> >
> > Hi Ubuntu Developers of the past, the present, and the future!
> >
> > For a while I'm working on an improvement to how Ubuntu
> > developers set up and handle their PGP keys. Without any
> > offense, up to now it mostly is an undocumented:
> > "Create a key, and somehow try to handle it safely".
> > But throughout the population of developers I've seen
> > various different interpretations of "safely" :-)
> >
> > Most of those that take it rather seriously have settled on a setup
> > that utilizes hardware keys and I was collecting their input and
> > experience for a while. I think it is time to drive a public policy
> > about what we would recommend.
> >
> > After some internal rounds with early adopters as well as internal
> > stakeholders on my draft, I've recently opened it up as a
> > public PR to the project docs [1] and already got quite good feedback
> > which is integrated by now. The intention is now to reach out
> > further, by pointing all of you to the PR [1].
> >
> > [1]: https://github.com/ubuntu/ubuntu-project-docs/pull/182
> >
> > P.S. I know there is more that can be done as subsequent steps in the future,
> > but I'm intentionally trying to not let future perfection be the
> > blocker of this helpful step today, like:
> >
> > - Testing and documenting exact steps to do that setup. For that I'd
> > want to get an agreement on the policy first, then distribute such
> > keys among some of our folks and ensure we polish any rough edges by
> > using them the way the policy says.
> >
> > - There are related aspects like the Launchpad API not yet having
> > such capabilities, we are pushing for that feature and we'd
> > adopt the policy here once available. I allude to that in the
> > presented PR, but until the capability exists I can't do much more.
> >
> > - It is considered to one day make some of this mandatory, at least for
> > roles with highly elevated permissions or e.g. within Canonical. But for
> > that we need to have the policy and steps (above), the way to set up
> > and use it (known missing) and maybe even missing launchpad features
> > agreed and implemented.
>
> I like the general idea of this, but please please PLEASE do not
> mandate the use of hardware-backed keys. Not all developers can
> actually obtain such keys (cost restrictions), some developers may
> reasonably distrust such keys (vulnerabilities in the embedded
> hardware of Yubikeys have been found before), and there are other ways
> to store a GPG key in a highly secure fashion, for instance by using
> an airgapped system, or (as I plan to do in the future) a Qubes OS
> machine with an airgapped vault VM.

Hi Aaron,
thanks for the answer, the appreciation of providing a good
set of recommendations, and the concerns.

I understand the concerns and agree with some of them, those are good
thoughts to discuss before anything becomes mandatory indeed.
And gladly I think a lot are mitigatable. As usual I think the best
option forward is probably some compromise in the middle.

Let me answer here directly and then ponder about how to get that into
the proposed documentation entry.

> Not all developers can actually obtain such keys (cost restrictions),

I think this is the easiest to overcome as I believe it is a project's
interest to get more secure.
IMHO if someone in a potential future is mandated to have a key
(unsure but IANAL maybe show they can't reasonably affort), they
should get them (funded) from the project.

Remember that in my suggestion for the future I considered this mostly
interesting for groups with higher permission.
Think small groups, maybe archive-admins to have an example.

And in a different angle of attack: If any business entity wants to
make this mandatory for their people contributing to the project, then
costs are a different story.
Think of Canonical as the best example, once establishing this policy,
I expect this to allow people to expense such keys.

> some developers may
> reasonably distrust such keys (vulnerabilities in the embedded
> hardware of Yubikeys have been found before), and there are other ways
> to store a GPG key in a highly secure fashion, for instance by using
> an airgapped system, or (as I plan to do in the future) a Qubes OS
> machine with an airgapped vault VM.

Counter arguments could be found to anything I guess :-/.
There might be things you distrust more, for example hw-keys can help
a lot in a case like "You distrust mandatory monitoring on your
company owned development system and want to harden against that".
And setting up a secondary airgapped system represents HW and time
investment as well - just like the HW-keys.
But these counter arguments do not matter much, do not raise your fist
against those and instead let us look at a result that acknowledge
imperfection.

I think we have to accept that any solution we might pick as
recommended will have potential vulnerabilities and drawbacks, and the
same is true for any alternative we could describe.
The past decades have shown the more popular a solution gets, the more
it is researched and attacked, eventually unveiling more issues.
And over enough time anything we'd pick in the past will be broken in
the future - if not by anything else then by raw compute power some
day.

Do not get me wrong - I think it is right to have a healthy distrust
of every solution, and revisit things like this suggested policy
regularly.
But I can't translate that into not recommending HW-keys at all either
and then iterating the same thought to eventually recommend none at
all.

In regard to the question worth being asked "but what If I'm in the
set of people distrusting the recommended solution more than
alternative X" I think the outcome should be to mention that
alternative setup X might be less common but is also considered to be
(I struggle for finding the right word to way what I want to say -
"allowed, tolerated, sufficient, ...?") sufficient.

Let me add an "alternatives" section to the end of my proposed text,
outlining how I think this could be integrated.
It would today start outlining the underlying dilemma, but point to no
alternative yet (Despite sharing some of your concerns, I feel unable
to fully describe an alternative setup I'd trust as much as HW keys).
You or others could over time describe and propose those alternatives
(since making them mandatory isn't anywhere close there is no rush to
that).
Eventually that would allow landing accepted alternatives in the same
documentation.
Let me try to prep that to show how it might look ...

[Way too many minutes later after rewriting it 34567897865678 times - arrr]
In the PR now, please have a look again

...

> Just my 2 cents as a MOTU. :)

Worth more than 2 cents, I highly appreciate the discussion!

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel