Feedback from the community -> ARM

[Uwe Kleine-König]

Hello,

next week there is a (virtual) meeting at ARM who invited some people
involved in Linux on ARM CPUs. One of the topics there is to tell them
Debian's needs and pain points.

My current list (based on own experience and asking for feedback in
#debian-arm) currently has:

- Fragmentation
- Vendor kernels vs. mainline
This got better in the past is my subjective impression, but it
still hurts. Device tree made this a tad simpler, but it's not
unusual to have vendor specific bindings.
- early boot code
U-Boot (or general: bootloader) is device specific and more often
than not there is only a Vendor variant available.
Also today there are more relevant components: ATF, UEFI/EDK2
Vendors care at different intensities (and profit from external
developers) Would Arm Base System Architecture (BSA) help? (This is
only for AArch64 though, arm32 still relevant for us.)
- relevant SoC/SBC vendors:
- Allwinner
- Broadcom / RaspberryPi Foundation
- Marvell
- NXP
- Odroid
- Rockchip
- some more for sure (which?)
- Graphics
Similar problematic, vendor blobs vs. OSS

Is there anything on your mind that is missing above and that you'd like
to be shared with ARM? Feel free to reply here or discuss in
#debian-arm. (I'm ukleinek there.)

Best regards
Uwe

[Marcin Juszkiewicz]

Post by Uwe Kleine-König
next week there is a (virtual) meeting at ARM who invited some people
involved in Linux on ARM CPUs. One of the topics there is to tell them
Debian's needs and pain points.
My current list (based on own experience and asking for feedback in
 - Fragmentation
   - Vendor kernels vs. mainline
     This got better in the past is my subjective impression, but it
     still hurts. Device tree made this a tad simpler, but it's not
     unusual to have vendor specific bindings.
   - early boot code
     U-Boot (or general: bootloader) is device specific and more often
     than not there is only a Vendor variant available.
     Also today there are more relevant components: ATF, UEFI/EDK2
   Vendors care at different intensities (and profit from external
   developers) Would Arm Base System Architecture (BSA) help? (This is
   only for AArch64 though, arm32 still relevant for us.)
   - Allwinner
   - Broadcom / RaspberryPi Foundation
   - Marvell
   - NXP
   - Odroid
   - Rockchip
   - some more for sure (which?)
 - Graphics
   Similar problematic, vendor blobs vs. OSS
Is there anything on your mind that is missing above and that you'd like
to be shared with ARM?

Sorry if it offends someone but I see Debian Arm team to be more Debian
Arm SBC team. Sure, it is most of available hardware as it includes
Windows-on-Arm laptops and Apple M1 systems too but Arm world has also
something outside of SBC.

My questions (probably get ignored):

1. When CBSA spec gets released? It is mentioned in BSA spec but
not public.
2. Are there plans to enforce BSA compliance for new designs?
3. How many years we need to wait for Arm systems to work out-of-
the-box? I mean unpack, connect input/video/power, boot generic
distro installer, install, reboot and use. So far there are
nearly no such ones outside of server space.

[Gene Heskett]

Post by Marcin Juszkiewicz

Post by Uwe Kleine-König
next week there is a (virtual) meeting at ARM who invited some
people involved in Linux on ARM CPUs. One of the topics there is to
tell them Debian's needs and pain points.
My current list (based on own experience and asking for feedback in
 - Fragmentation
   - Vendor kernels vs. mainline
     This got better in the past is my subjective impression, but
it still hurts. Device tree made this a tad simpler, but it's not
unusual to have vendor specific bindings.
   - early boot code
     U-Boot (or general: bootloader) is device specific and more
often than not there is only a Vendor variant available.
     Also today there are more relevant components: ATF, UEFI/EDK2
   Vendors care at different intensities (and profit from external
   developers) Would Arm Base System Architecture (BSA) help? (This
is only for AArch64 though, arm32 still relevant for us.)
   - Allwinner
   - Broadcom / RaspberryPi Foundation
   - Marvell
   - NXP
   - Odroid
   - Rockchip
   - some more for sure (which?)
 - Graphics
   Similar problematic, vendor blobs vs. OSS
Is there anything on your mind that is missing above and that you'd
like to be shared with ARM?

Sorry if it offends someone but I see Debian Arm team to be more
Debian Arm SBC team. Sure, it is most of available hardware as it
includes Windows-on-Arm laptops and Apple M1 systems too but Arm world
has also something outside of SBC.
1. When CBSA spec gets released? It is mentioned in BSA spec but
not public.

New acronym, please define.

Post by Marcin Juszkiewicz
2. Are there plans to enforce BSA compliance for new designs?
3. How many years we need to wait for Arm systems to work out-of-
the-box? I mean unpack, connect input/video/power, boot generic
distro installer, install, reboot and use. So far there are
nearly no such ones outside of server space.

This is also true, and a bit of a sore spot with me. Why? Because the pi
folks furnish only the bare 4.19-something realtime kernel, only in
source form direct from kernel.org if you want a real-time system,
needed to run something like LinuxCNC, which is capable of running 99%
of the CNC machines that make your fawncy toys on this planet. And with
only a few exceptions, makeing product faster than the the machines
original desgner had in mind.

They are snotty as hell, on their forum and will not offer any help in
configureing, building, or installing that kernel. Questions are
ignored, and if you ping the forum again, you will get invited to take a
long walk off a short pier.

Because the pi's u-boot is configured differently from anybody else's, I
spent a couple months studying how to install that kernel once it was
built, and finally invented my own installer which has worked well now
for over a year, on several more machines now, a very simple installer
that unpacks a less than 30 meg tarball preconfigured to copy whats
needed to the proper locations on the pi's u-sd boot card while its
mounted in a card reader. And it Just Works. But it has been far from
painless.

Cheers, Gene Heskett

[Marcin Juszkiewicz]

Post by Gene Heskett

Post by Marcin Juszkiewicz
1. When CBSA spec gets released? It is mentioned in BSA spec but
not public.

New acronym, please define.

I can not as CBSA is all you have in public docs.

It is "C* Base System Architecture" probably where "C*" can be any word.
If you have NDAs signed with Arm then reach there and ask.

Post by Gene Heskett

Post by Marcin Juszkiewicz
2. Are there plans to enforce BSA compliance for new designs?
3. How many years we need to wait for Arm systems to work out-of-
the-box? I mean unpack, connect input/video/power, boot generic
distro installer, install, reboot and use. So far there are
nearly no such ones outside of server space.

This is also true, and a bit of a sore spot with me. Why? Because the pi
folks

To be honest: I do not care how bad Arm SBCs are. I just skip them and
do not plan to spend any money on buying those. Most are really far from
being usable as generic Arm system.

[Gene Heskett]

Post by Marcin Juszkiewicz

Post by Gene Heskett

Post by Marcin Juszkiewicz
1. When CBSA spec gets released? It is mentioned in BSA spec but
not public.

New acronym, please define.

I can not as CBSA is all you have in public docs.
It is "C* Base System Architecture" probably where "C*" can be any
word. If you have NDAs signed with Arm then reach there and ask.

Post by Gene Heskett

Post by Marcin Juszkiewicz
2. Are there plans to enforce BSA compliance for new designs?
3. How many years we need to wait for Arm systems to work out-of-
the-box? I mean unpack, connect input/video/power, boot
generic distro installer, install, reboot and use. So far there are
nearly no such ones outside of server space.

This is also true, and a bit of a sore spot with me. Why? Because
the pi folks

To be honest: I do not care how bad Arm SBCs are. I just skip them and
do not plan to spend any money on buying those. Most are really far
from being usable as generic Arm system.

Same for rockchip. Armbian runs great on it, but what can it really do?
Docs suck at lest as bad as the pi's.

Cheers, Gene Heskett

[Diederik de Haas]

Post by Marcin Juszkiewicz
3. How many years we need to wait for Arm systems to work out-of-
the-box? I mean unpack, connect input/video/power, boot generic
distro installer, install, reboot and use. So far there are
nearly no such ones outside of server space.

+1
What an excellent description of what I tried to convey (probably poorly) on
IRC about how horrible and complicated a first-use experience is.

Post by Marcin Juszkiewicz
I see Debian Arm team to be more Debian Arm SBC team.

I'm guessing there's much requests wrt SBCs as that is for many people their
first experience with ARM systems (besides phones). It is/was for me.
And they make for (potentially) great fits for systems that need to be on 24/7,
like quassel-core or FreedomBox (a Debian Pure Blend), as they consume little
power, which is great for the environment and your wallet.

[Ralph Aichinger]

Post by Diederik de Haas
I'm guessing there's much requests wrt SBCs as that is for many people their
first experience with ARM systems (besides phones). It is/was for me.
And they make for (potentially) great fits for systems that need to be on 24/7,
like quassel-core or FreedomBox (a Debian Pure Blend), as they
consume little
power, which is great for the environment and your wallet.

Absolutely! It used to be that people installed Linux on old PCs,
but for ecological reasons (power consumption) and practical ones
(many PCs today are notebooks and they are often no longer replaced
while basically working, but often their failure mode is being
completely dead after a drop etc.) this is much less attractive
than even 10 years ago.

I really think Debian should have a better answer to installing
on the Raspberry Pi, as this ist the only board that is widely
available, sold in *huge* numbers (40 million?), can boot aarch64,
has up to 8GB RAM (which makes it quite usable for many tasks),
and most of all a long support times (they guarantee production of
the RPi4 until January 2026, which is probably longer than most
Intel hardware sold today, and probably four times as long as
comparable SBC products.

I really don't know if the UEFI/ACPI install path for the RPi4 is
preferable or some other way, but the Debian project should be
more explicit and clear in suggesting a way to install aarch64
on the RPi4. Many criticisms of the RPi that were true 5 years
ago no longer hold. How much different is the process of booting
an RPi 4 with UEFi from e.g. booting some run of the mill notebook
with obscure Realtek components that need binary blobs too?

As somebody who has used Raspbian now for years, quassel-core
or FreedomBox is rather offputting, because I very much prefer
true Debian that matches amd64 except for architecture. If I
wanted to have some distribution that is "Debian based" and
not Debian outright, I'd probably go for Raspberry Pi OS.

I have installed Debian on a SGI Indy, some Digital Alpha of
sorts and a Sparcstation ELC (I think) in the past, but official
documentation for these even then rather niche machines was much
much clearer than the official documentation is for installing
on the Raspberry Pi, a board that is sold by the million and
within the reach of schoolchildren.

The official arm64 install documentation lists as of today
as supported arm64 boards:

Applied Micro (APM) Mustang/X-Gene T
ARM Juno Development Platform

Has anybody of you seen these in the wild?
Why is the Raspberry Pi 4 with UEFI or the stock boot process
not listed here?

https://www.debian.org/releases/bullseye/arm64/install.en.pdf
(page 6)

What can a mere Debian user like me do to improve this documentation
problem?

/ralph

[Marcin Juszkiewicz]

I really think Debian should have a better answer to installing on
the Raspberry Pi, as this ist the only board that is widely
available, sold in *huge* numbers (40 million?), can boot aarch64,
has up to 8GB RAM (which makes it quite usable for many tasks), and
most of all a long support times (they guarantee production of the
RPi4 until January 2026, which is probably longer than most Intel
hardware sold today, and probably four times as long as comparable
SBC products.

Ask Raspberry/Pi vendor then. Make them work on getting support for
their product into mainline, make use of available standards during
design of their next products.

Properly designed Arm board should boot generic ISO written into USB
stick (dd if=d-i.iso of=/dev/sda). Several SBC work that way already (or
can have their firmware flashed to be that way).

Distributions should not waste time on getting SBC systems working but
SBC vendors should make them 'just work' with distributions.

You go to store, buy x86-64 laptop/computer and expect it to just work
with Debian. Arm hardware should work same way. Just vendors do not care
because users are so used to 'shit, this arm device needs some special
treatment' way of support.

How much different is the process of booting an RPi 4 with UEFi from
e.g. booting some run of the mill notebook with obscure Realtek
components that need binary blobs too?

'of the mill notebook': connect cables, plug USB stick with generic
installer, boot, install, use. Then use the same USB stick with another
notebook from different vendor.

RPi4: find out how it boots, prepare SD card with RPi4 specific
installer, boot, install, use. And then rewrite SD card for another SBC.

I see a difference.

As somebody who has used Raspbian now for years, quassel-core or
FreedomBox is rather offputting, because I very much prefer true
Debian that matches amd64 except for architecture. If I wanted to
have some distribution that is "Debian based" and not Debian
outright, I'd probably go for Raspberry Pi OS.

Let me be honest: if someone wants to use R/Pi SBC then R/Pi OS is what
they should use. To not waste time of other distribution maintainers.

This way you get 'probably working' kernel with all out-of-tree patches
needed to get board working and functional + set of packages with
out-of-tree patches to get userspace running.

Debian, Fedora and other distros usually do not ship those out-of-tree
patches because they are not even on a way to mainline projects.

The official arm64 install documentation lists as of today as
Applied Micro (APM) Mustang/X-Gene T ARM Juno Development Platform
Has anybody of you seen these in the wild?

I booted bullseye on Mustang during last week.

Why is the Raspberry Pi 4 with UEFI or the stock boot process not
listed here?

Because no one contributed it?

What can a mere Debian user like me do to improve this documentation
problem?

Check doc, write new one, send to maintainer. Or even to this ML.

[Ralph Aichinger]

Post by Marcin Juszkiewicz
Ask Raspberry/Pi vendor then. Make them work on getting support for
their product into mainline, make use of available standards during
design of their next products.

Ist this criticism fair?

I am running on my aarch64 RPi4 currently:

***@pi:~# uname -a
Linux pi 5.10.0-7-arm64 #1 SMP Debian 5.10.40-1 (2021-05-28) aarch64
GNU/Linux
***@pi:~# dmesg | head -4
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[ 0.000000] Linux version 5.10.0-7-arm64
(debian-***@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1
20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian
5.10.40-1 (2021-05-28)
[ 0.000000] efi: EFI v2.70 by https://github.com/pftf/RPi4
[ 0.000000] efi: ACPI 2.0=0x33a20018 SMBIOS=0x37100000 SMBIOS
3.0=0x370e0000 MEMATTR=0x358cc018 MOKvar=0x358ce000 RNG=0x372db798
MEMRESERVE=0x33682118
***@pi:~# dpkg -l "*5.10.0-7*"
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-
aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture
Description
+++-===================================-============-============-
=============================================
ii linux-image-5.10.0-7-arm64 5.10.40-1 arm64 Linux
5.10 for 64-bit ARMv8 machines (signed)

a stock Debian kernel, that does everything I need (USB mass storage
for disk, GigE for network, HDMI for install). It has no WiFi, but
I have had lots of arm64 hardware that did not either.

Post by Marcin Juszkiewicz
Properly designed Arm board should boot generic ISO written into USB
stick (dd if=d-i.iso of=/dev/sda). Several SBC work that way already
(or can have their firmware flashed to be that way).

Yes, this would be preferrable, but booting via EFI partition
is not that bad either. And can be considered a standard way
of booting too. Having to copy the files to FAT32 is an inconvenience,
but on the other hand ISO9660 is an obsolete inconvenience too
(wo installs from real, rotating physical media today?).
BTW I've had quite some problems getting even run of the mill
PC hardware boot from USB media into UEFI. Tianocore for Pi
has really become quite good.

Post by Marcin Juszkiewicz
Distributions should not waste time on getting SBC systems working
but SBC vendors should make them 'just work' with distributions.

"Should", yes. But I very rarely hear the same criticism concerning
Asus mainboards or Dell notebooks, when they do not consider
Linux distributions that much either.

Post by Marcin Juszkiewicz
You go to store, buy x86-64 laptop/computer and expect it to just
work with Debian.

Have you tried that recently? Bought some arbitrary laptop without
knowing detailed lists of included components, and inserted a
netinstall USB key? Chances are good this won't work that much
better than installing on RPi4, because you need at least
firmware-realtek (non-free) to get any networking at all.

Post by Marcin Juszkiewicz
'of the mill notebook': connect cables, plug USB stick with generic
installer, boot, install, use. Then use the same USB stick with
another notebook from different vendor.

And get no networking for netinstall? The last 4 or 5 amd64
computers I installed for myself (i.e. not dedicated server
hardware, that is *a lot* better) needed some non-free drivers,
making install just about as convenient as on the RPi4. Mainly
it was for networking (both wired and WiFi).

Post by Marcin Juszkiewicz
RPi4: find out how it boots, prepare SD card with RPi4 specific
installer, boot, install, use. And then rewrite SD card for another SBC.
I see a difference.

So do I. "find out how it boots" is much more complicated with
the RPi4, as there is very little official Debian documentation
on this topic. There are probably 4 or 5 ways to boot the Pi 4,
and no official guidance which to pick or which are considered
supported or better supported ways of installing.

I have installed on Sparcstations in the past, and install
was complicated (IIRC bootp/tftp netboot, ...) but at least
there was official documentation on how to get your Sparc machine
up. In the official release notes, and not by googling various
tutorials of variyng degrees of trustworthyness.

This documentation is sorely missing for the Pi today. And Sun
did not care too much about Debian back then either.

Post by Marcin Juszkiewicz
Let me be honest: if someone wants to use R/Pi SBC then R/Pi OS is
what they should use. To not waste time of other distribution
maintainers.

And if I buy an Asus mainboard, then I should use Windows and
not waste the time of distribution maintainers? What about the
other official Debian ports?

What I want to say is not that I think I am entitled to ask
anything from Debian porters and developers (I am certainly not),
but the double standard of accepting that Asus, Dell, Acer, etc.
could not care less about Debian support for non-server/workstation
class hardware, and not even documenting how to best install
on Raspberry Pi (yes, the Raspberry Pi foundation probably does not
care too much about Debian either) surprises me a bit.

Is there a way to help here?

There are probably about 10 million RPi4+400 machines out by now
and only about 400 aarch64 entries in popcon. Why is that? To me that
is a missed opportunity. I read lots and lots of postings about
people installing Kubernetes setups on 5 or so Pi4 machines (aarch64)
because the pi fills a niche there (cheap, but also good value for
money). Why should these people all be steered towards Ubuntu? And no,
I don't imagine these people going out and buying Mustang ARM boards.

Post by Marcin Juszkiewicz
This way you get 'probably working' kernel with all out-of-tree
patches needed to get board working and functional + set of packages
with out-of-tree patches to get userspace running.

My Pi 4 runs an unpatched arm64 kernel directly from Debian. To me it
seems completely functional except for missing WiFi drivers in my
headless setup. It runs for months without any problems. What am I
missing?

On the other hand I often hear people recommending obscure SBCs
that only boot with very specific, old kernels, *over the Raspberry
Pi*. Most RPi patches have been upstreamed before 5.10 or so
(luckily for bullseye), with WiFi in 5.12 AFAIK.

Post by Marcin Juszkiewicz
Debian, Fedora and other distros usually do not ship those out-of-
tree
patches because they are not even on a way to mainline projects.

These patches are basically no longer needed for the Pi4, and
other distributions (Ubuntu, Fedora) are much much easier to
install on the pi.

Post by Marcin Juszkiewicz
Check doc, write new one, send to maintainer. Or even to this ML.

I will try.

/ralph

[Diederik de Haas]

Post by Ralph Aichinger
I really think Debian should have a better answer to installing
on the Raspberry Pi

I think the problem isn't on Debian's side, but on the side of the Raspberry
Pi Foundation (RPF).
If they would upstream more/sooner or better yet, use an upstream first
approach, the whole (FLOSS) world, including Debian, would benefit.
But for some reason this is still problematic :-/

Post by Ralph Aichinger
can boot aarch64

Almost entirely because of Debian and people (outside RPF) working with
upstream (kernel) to make it work on arm64. Which then also benefits RPF.

Post by Ralph Aichinger
As somebody who has used Raspbian now for years, quassel-core
or FreedomBox is rather offputting, because I very much prefer
true Debian that matches amd64 except for architecture. If I
wanted to have some distribution that is "Debian based" and
not Debian outright, I'd probably go for Raspberry Pi OS.

quassel-core and FreedomBox are (a collection of) programs already available
in Debian, which you can install with apt[itude|-get].
The FreedomBox project does also provide downloadable images, but that just/
still a (pure/true) Debian system.

So I guess there's some miscommunication/misunderstanding here?

[Ralph Aichinger]

Post by Diederik de Haas
I think the problem isn't on Debian's side, but on the side of the
Raspberry Pi Foundation (RPF).

But if it is, why is e.g. installing Ubuntu so much easier
(as opposed to amd64 where it is about the same) and so much
better documented?

Post by Diederik de Haas
quassel-core and FreedomBox are (a collection of) programs already
available in Debian, which you can install with apt[itude|-get].
The FreedomBox project does also provide downloadable images, but
that just/still a (pure/true) Debian system.

But I still would have to research if e.g. the installer leaves
some settings differently, if e.g. /etc/issue is different from
Debian, if there are any cosmetic differences (many derived
distributions e.g. change default editors, or shell prompts,
graphic themes etc.). These are small things, but to me they
really matter. E.g. Raspbian decided a few years ago to change
default Debian dhpc client behaviour. Nothing hugely problematic,
but just one more difference (even if well-intended) to think
about. If I can choose, I really want actual, unchanged Debian
and I do not deny that true blends have their place.

Post by Diederik de Haas
So I guess there's some miscommunication/misunderstanding here?

Maybe. To me the main thing is: I want Debian arm64. If installing
$SOMETHING (like FreedomBox) gets me that: Why not, but if I am
not sure in what aspects it differs from Debian afterwards, then I'd
rather go the way of installing Debian directly, even if it is a bit
more complicated and/or not as well documented.

/ralph

[Andrei POPESCU]

Post by Ralph Aichinger
The official arm64 install documentation lists as of today
Applied Micro (APM) Mustang/X-Gene T
ARM Juno Development Platform
Has anybody of you seen these in the wild?
Why is the Raspberry Pi 4 with UEFI or the stock boot process
not listed here?
https://www.debian.org/releases/bullseye/arm64/install.en.pdf
(page 6)
What can a mere Debian user like me do to improve this documentation
problem?

Specifically for the Release Notes a bug against the package
'release-notes', preferably with a patch (or merge request in Salsa), or
at least suggested wording.

Other documentation has similar processes (if in doubt ask on
debian-doc), except for the Wiki, which anyone should be able to edit
directly.

Kind regards,
Andrei

[Paul Wise]

Many criticisms of the RPi that were true 5 years ago no longer hold.

Some of them are still true; the weird GPU-starts-CPU SoC boot
process, blobs required for the boot process, some Linux kernel
patches are not in mainline etc

or FreedomBox is rather offputting, because I very much prefer
true Debian that matches amd64 except for architecture.

FreedomBox is a Debian blend, so a FreedomBox install *is* a Debian
install, there are no custom packages or other hacks. At least that is
how it is claimed to be, I haven't tried it.

Why is the Raspberry Pi 4 with UEFI or the stock boot process
not listed here?

I assume because of the proprietary software needed to boot the RPi4,
although there is a WIP libre replacement that isn't yet in Debian,
see other comments in the thread about that.

[Ralph Aichinger]

Post by Paul Wise

Many criticisms of the RPi that were true 5 years ago no longer hold.

Some of them are still true; the weird GPU-starts-CPU SoC boot
process,

Is this still true for the Raspberry Pi4 with UEFI?

https://github.com/pftf/RPi4

Even Debian Wiki says

https://wiki.debian.org/RaspberryPi

"All Raspberry Pi models before the 4 (1A, 1B, 1A+, 1B+, Zero, Zero W,
2, 3) boot from their GPU"

So it seems this is no longer true, and exactly what I said.

Post by Paul Wise
blobs required for the boot process,

If there are Blobs needed to bring up the RPi4 they are included
in above UEFi firmware (or the stuff used in other boot mechanisms).
I don't see how this is different from the "blobs" I load when I
boot my UEFi Asus mainboard.

Post by Paul Wise
some Linux kernelpatches are not in mainline etc

What patches do you mean? Patches that are in Debian kernels, but
not in Linus' upstream kernels? Or patches on top of what Debian
does? Because if you mean the latter, than I can assure you, that
the RPi4b runs fine without anything but a stock debian kernel,
both during installation (there the kernel from recent bullseye arm64 
netinst works just fine) as in actual use. The kernel I am currently
running is

ii linux-image-5.10.0-7-arm64 5.10.40-1 arm64 Linux
5.10 for 64-bit ARMv8 machines (signed)

directly from Debian archive.

And I stand by my statement: What was true 5 years ago (weird GPU boot,
patched proprietary kernel, architecture that falls right between
armel/armhf in Debian) is no longer true for kernels after 5.10
and with RPi4. The RPi4 works with stock Debian kernels (and supposedly
stock upstream kernels), runs straight arm64 just like other devices
and boots debian netinstall installation directly as long as you copy
it over onto an UEFi partiton with the UEFi EDK2 files included.

Post by Paul Wise
FreedomBox is a Debian blend, so a FreedomBox install *is* a Debian
install, there are no custom packages or other hacks. At least that
is how it is claimed to be, I haven't tried it.

I have not tried it either, but I am slightly worried about
small variations in the details. And I do not need the extra
software they install (yes, I could uninstall, of course).

My most recent Pi4 is used for taking backups with restic over
wireguard, so I more or less just need restic, wireguard, and
ssh (plus the usual commandline utilities). It would feel wrong
to install Freedombox, look what has to be uninstalled, look
for potential specific configurations etc.

Post by Paul Wise
I assume because of the proprietary software needed to boot the
RPi4,a lthough there is a WIP libre replacement that isn't yet in
Debian, see other comments in the thread about that.

With "WIP libre replacement" do you mean the tianocore/EDK2/UEFi port
here?

https://github.com/pftf/RPi4

Or something else? 

/ralph

[Paul Wise]

Post by Ralph Aichinger
"All Raspberry Pi models before the 4 (1A, 1B, 1A+, 1B+, Zero, Zero W,
2, 3) boot from their GPU"
So it seems this is no longer true, and exactly what I said.

I'm fairly sure that the RPi4 still boots via the VC4 chip, but I
cannot find any proper documentation of the RPi4 SoC boot ROM like one
can normally find with other SoCs. The official docs only say that the
boot process is no different to older RPi hardware with the exception
of there being an EEPROM involved.

https://github.com/raspberrypi/documentation/blob/master/hardware/raspberrypi/bootmodes/bootflow_2711.md
https://hackaday.io/page/6372-raspberry-pi-4-boot-sequence

Post by Ralph Aichinger
If there are Blobs needed to bring up the RPi4 they are included
in above UEFi firmware (or the stuff used in other boot mechanisms).
I don't see how this is different from the "blobs" I load when I
boot my UEFI Asus mainboard.

Correct, that is no different, but still suboptimal and can be replaced
with libre software (for eg coreboot) on some machines.

Post by Ralph Aichinger

some Linux kernel patches are not in mainline etc

What patches do you mean?

WRT the Linux kernel, mainline always means kernel.org.

Post by Ralph Aichinger
the RPi4b runs fine without anything but a stock debian kernel

That may be true, but there are apparently patches being maintained by
RPi that aren't being mainlined, I don't know the details though.

Post by Ralph Aichinger
With "WIP libre replacement" do you mean the tianocore/EDK2/UEFi port
here?

Sorry, I misremembered where it was mentioned and it turns out to be
another thread on another list:

https://lists.debian.org/debian-devel/2021/06/msg00043.html

I mean rpi-open-firmware:

https://github.com/librerpi/rpi-open-firmware/
https://github.com/itszor/vc4-toolchain
https://github.com/itszor/vc4-toolchain/issues/7

BTW, the RPi4 boot process even has some DRM in it, but that is
apparently easy to bypass due to bugs in boot code:

https://github.com/librerpi/rpi-open-firmware/blob/master/docs/cracking-rpi4-hmac.txt

Post by Ralph Aichinger
https://github.com/pftf/RPi4

Will this be added to edk2-platforms?

https://github.com/tianocore/edk2-platforms/

[Pete Batard]

Post by Paul Wise
Will this be added to edk2-platforms?
https://github.com/tianocore/edk2-platforms/

It already has:

https://github.com/tianocore/edk2-platforms/tree/master/Platform/RaspberryPi/RPi4

The Raspberry Pi 4 UEFI firmware has been an official EDK2
implementation for some time...

As a matter of fact, starting with RPi3, the Raspberry Pi has been an
official EDK2/UEFI platform for more than 2 years now.

As to libre replacements for the proprietary blobs, while this is of
course a desirable effort that I too hope can come to fruition, it
wouldn't change much in terms of booting a generic kernel from UEFI
since those blobs intervene way sooner than that. Even with libre
versions of those the boot handover process on the Pi would still be
blobs -> TFA -> UEFI -> kernel, which is how you want it when you do
have the luxury of an ARM64 platform with a full blow UEFI firmware.

Also, AFAIK, coreboot still has to use proprietary blobs to bring up RAM
and other stuff that intel/AMD don't want to open up, which they've
pretty much come to terms with, so I'm not sure using coreboot as an
example to follow, for doing away with proprietary blobs, is that
effective. Similar to coreboot, the UEFI firmware we have on the Pi 4
should actually be construed as a way to keep the annoying proprietary
stuff away, and instead provide an Open Source implementation for one of
the rare well established industry standards that the mainline kernel,
and by extension Debian, could actually rely on to *simplify* their
operations.

I mean, isn't the goal of being able to stop chasing after yet another
custom implementation, in order to support this or that ARM platform,
one of the points being sought from this discussion? Because if that is
the case, then, instead of demanding that ARM SoC/SBC manufacturers do
away with proprietary blobs, which is unlikely to happen, Debian could
do itself a favour by embracing UEFI support for platforms that have it
and at the same time try to set a clear delimiter of responsibilities
("ARM manufacturers, if you provide a working UEFI implementation for
your platform to take care of the early boot code/custom kernel
headaches, then we'll see what we can do to make our distribution work
on it").

Regards,

/Pete

[Wookey]

Post by Pete Batard
As a matter of fact, starting with RPi3, the Raspberry Pi has been an
official EDK2/UEFI platform for more than 2 years now.
... Debian could do itself a
favour by embracing UEFI support for platforms that have it and at the same
time try to set a clear delimiter of responsibilities ("ARM manufacturers,
if you provide a working UEFI implementation for your platform to take care
of the early boot code/custom kernel headaches, then we'll see what we can
do to make our distribution work on it").

We have embraced UEFI and if it's available the installer should use it.

It is my understanding that the current (testing) vanilla debian
installer does boot on a UEFI Pi4 (after we fixed #985956). Is that understanding wrong?

Wookey

[Andrew M.A. Cater]

Post by Wookey

Post by Pete Batard
As a matter of fact, starting with RPi3, the Raspberry Pi has been an
official EDK2/UEFI platform for more than 2 years now.
... Debian could do itself a
favour by embracing UEFI support for platforms that have it and at the same
time try to set a clear delimiter of responsibilities ("ARM manufacturers,
if you provide a working UEFI implementation for your platform to take care
of the early boot code/custom kernel headaches, then we'll see what we can
do to make our distribution work on it").

We have embraced UEFI and if it's available the installer should use it.
It is my understanding that the current (testing) vanilla debian
installer does boot on a UEFI Pi4 (after we fixed #985956). Is that understanding wrong?

Hi Wookey - long time no see in person.

I can confirm that the Bullseye Debian arm64 installer works perfectly with
UEFI. Following the instructions suggested by Pete Batard, I installed a
4G RPi 4 today with an SSD as the main drive rather than an SD card / USB
drive.

All tthe best, as ever,

Andy Cater

Post by Wookey
Wookey
--
Principal hats: Linaro, Debian, Wookware, ARM
http://wookware.org/

[Pete Batard]

Post by Wookey

Post by Pete Batard
As a matter of fact, starting with RPi3, the Raspberry Pi has been an
official EDK2/UEFI platform for more than 2 years now.
... Debian could do itself a
favour by embracing UEFI support for platforms that have it and at the same
time try to set a clear delimiter of responsibilities ("ARM manufacturers,
if you provide a working UEFI implementation for your platform to take care
of the early boot code/custom kernel headaches, then we'll see what we can
do to make our distribution work on it").

We have embraced UEFI and if it's available the installer should use it.

Yes, and I appreciate your help with that.

But from my perspective, it was quite a struggle to get a small patch,
that was critical for proper UEFI installation, to be looked into by
Debian folks with some priority. And it's not until you stepped in that
things moved forward. Plus it appears that we were also lucky that the
Bullseye release was delayed, else that patch, and thus the ability to
install Bullseye on the Pi 4 in UEFI mode, may not have made it into the
release.

Granted we also should have logged a bug for that issue as soon as we
picked it up, so we have to share part of the blame on this one. But the
delays in processing #985956, even though we tried to bring attention
over and over again to the fact that this very negatively affected one
of the rare UEFI installation of Bullseye on ARM64, on what has to be
the current most widespread system for that arch, was a bit concerning.
Especially, it is not the first time we've seen what I would qualify as
extended delays in trying to get showstopping UEFI patches looked into.
As a matter of fact, I eventually gave up trying to get the necessary
Pi4 network patches backported into Debian 10 (#950578), because even
though we did submit a working patch from the get go, it took 3 months
(!) to actually get someone on the Debian side to look into it, only to
then tell us to just go re-do that patch, which is not what I would
qualify as a viable mode of operation.

So, at the risk of being controversial, it doesn't seem to me like
everybody in the Debian world has been that willing to embrace UEFI. Or
if they do, then not everybody appears to have recognized the importance
of being able to provide Debian in UEFI mode on what has to account for
the most popular ARM64 platform. And I could point to further evidence
of this when I also brought the idea on this list that, maybe, it was
time for Debian to start looking into moving away from using good old
uboot or similar for distributing special installable images for
platforms like the Pi not that we have a full blown UEFI, as this very
idea seemed to irk a few people (which, to be fair, may not have been
directly affiliated with Debian).

But regardless of this, my remark was aimed more at the topic at hand
which should be what feedback Debian may want to convey to the ARM
community, and especially the ARM platform manufacturers, and therefore
goes beyond than just the Raspberry Pi platform.

In short, I would reiterate that, in light of the headaches caused by
all the other approaches, and especially a requirement that still seems
to boil down to providing custom images (in part because of custom boot
methods as well as proprietary blobs, which I don't realistically see as
going away) the feedback Debian should provide to ARM vendors is that,
if the latter want better cooperation with Linux distros, then they
should put the onus on releasing a working UEFI firmware for their
platform, especially as it's been demonstrated that, even for an SBC as
quirky and as ill-suited for it as the Raspberry Pi (and that was part
of the point of the whole exercise), it is not that difficult to achieve.

Regards,

/Pete

[Marcin Juszkiewicz]

Post by Pete Batard
So, at the risk of being controversial, it doesn't seem to me like
everybody in the Debian world has been that willing to embrace UEFI.

Debian is one of distributions supporting UEFI on AArch64 from start of
this port (2013).

Just most of people use it on SBC where U-Boot is a king and many of
them use U-Boot in old way with boot.scr and other scripted methods
rather than using EFI services.

Post by Pete Batard
And I could point to further evidence of this when I also brought the
idea on this list that, maybe, it was time for Debian to start
looking into moving away from using good old uboot or similar for
distributing special installable images for platforms like the Pi

Let me bring the idea on this list that, maybe, it is time for vendors
to start looking into moving away from using good old U-Boot on microsd
or similar to provide on-board SPI flash to store bootloader. So we
could stop distributing special installable images for platforms like
the Pi.

[here goes few other ideas written same way to show that it is not
distro which should adapt to sick market but vendors should fix the
broken market]

https://marcin.juszkiewicz.com.pl/2021/02/15/ebbr-on-espressobin/

https://marcin.juszkiewicz.com.pl/2020/06/17/ebbr-on-rockpro64/

[Paul Wise]

Post by Marcin Juszkiewicz
Let me bring the idea on this list that, maybe, it is time for vendors
to start looking into moving away from using good old U-Boot on microsd
or similar to provide on-board SPI flash to store bootloader. So we
could stop distributing special installable images for platforms like
the Pi.

Storing boot firmware on the board seems to encourage various
sub-optimal things. Fully proprietary bootloaders. GPL violating
bootloaders. Bootloaders containing blobs. Bootloaders that are locked
down and cannot be replaced at all, especially on mobile. Forks of
random snapshots of existing bootloaders.

Personally I would go the other way; there should be a standard boot
protocol between the SoC boot ROM and later layers of the boot process
and vendors should stop shipping any boot firmware beyond the SoC boot
ROM, leaving it to OS distributors to package mainline bootloaders
that support that protocol.

[Marcin Juszkiewicz]

Post by Paul Wise

Post by Marcin Juszkiewicz
Let me bring the idea on this list that, maybe, it is time for vendors
to start looking into moving away from using good old U-Boot on microsd
or similar to provide on-board SPI flash to store bootloader. So we
could stop distributing special installable images for platforms like
the Pi.

Storing boot firmware on the board seems to encourage various
sub-optimal things. Fully proprietary bootloaders. GPL violating
bootloaders. Bootloaders containing blobs. Bootloaders that are locked
down and cannot be replaced at all, especially on mobile. Forks of
random snapshots of existing bootloaders.
Personally I would go the other way; there should be a standard boot
protocol between the SoC boot ROM and later layers of the boot process
and vendors should stop shipping any boot firmware beyond the SoC boot
ROM, leaving it to OS distributors to package mainline bootloaders
that support that protocol.

Whatever allow SBC to have "unpack, connect, install, reboot, use".

Now on-boot-media bootloader can be whatever crap you can imagine.

And still requires maintaining all those /InstallDebianOn/RandomSBC wiki
pages, tools to generate installer images per board, tools to generate
bootable images per board etc.

Choose your poison?

[Arnd Bergmann]

On Mon, Jun 14, 2021 at 10:45 AM Marcin Juszkiewicz

Post by Marcin Juszkiewicz

Post by Paul Wise

Post by Marcin Juszkiewicz
Let me bring the idea on this list that, maybe, it is time for vendors
to start looking into moving away from using good old U-Boot on microsd
or similar to provide on-board SPI flash to store bootloader. So we
could stop distributing special installable images for platforms like
the Pi.

As far as I can tell (please let me know if this has changed), the only
way to get UEFI boot on Raspberry Pi is to load an EDK2 from
a special partition on a bootable drive (µSD or USB). As you still
need device specific boot media, it does not solve the problem that
BBR tries to address by asking for UEFI support in the boot
firmware.

Post by Marcin Juszkiewicz

Post by Paul Wise
Storing boot firmware on the board seems to encourage various
sub-optimal things. Fully proprietary bootloaders. GPL violating
bootloaders. Bootloaders containing blobs. Bootloaders that are locked
down and cannot be replaced at all, especially on mobile. Forks of
random snapshots of existing bootloaders.
Personally I would go the other way; there should be a standard boot
protocol between the SoC boot ROM and later layers of the boot process
and vendors should stop shipping any boot firmware beyond the SoC boot
ROM, leaving it to OS distributors to package mainline bootloaders
that support that protocol.

Whatever allow SBC to have "unpack, connect, install, reboot, use".
Now on-boot-media bootloader can be whatever crap you can imagine.
And still requires maintaining all those /InstallDebianOn/RandomSBC wiki
pages, tools to generate installer images per board, tools to generate
bootable images per board etc.

Agreed, it is impossible to have board-agnostic boot media at the same
time as board-agnostic boot ROM unless you have some other place to
store board specific code and data, pretty much by definition. Whether
you store this in eMMC boot-partition, a spi-nor chip, or on-chip flash
doesn't matter too much, but most boards already have at least on of
those. The missing bit that I see is getting board vendors to actually enable
UEFI support in their boot firmware, and storing (all of) it in the boot
partition rather than the eMMC data partition.

Arnd

[Pete Batard]

Post by Arnd Bergmann
As far as I can tell (please let me know if this has changed), the only
way to get UEFI boot on Raspberry Pi is to load an EDK2 from
a special partition on a bootable drive (µSD or USB).

Not really.

You can load everything from a standard FAT32 EFI System Partition
(ESP), as least on the Raspberry Pi 4, which allows you to create media
that is 100% UEFI compliant from the get go (no need for special
partitions or anything) and also has the benefit of allowing you to
install vanilla Debian using the same media for the target and the
source (without even having to do any extra gymnastics for reboot),
which can be a hugely beneficial thing on SBCs where connectivity might
be limited or where users may not have the luxury of being able to
afford 2 media.

Post by Arnd Bergmann
Create a GPT ESP (EFI System Partition) onto an USB, extract the
netinst.iso content there, add the latest Raspberry PI UEFI firmware
and proceed to a standard Debian networked installation. That's it!

And by the way, this method of installing Debian from a single media by
leveraging the ESP is something that's also frequently used on x86 UEFI
based PCs, so there's really nothing "custom" about that method, apart
from the fact that you need to sprinkle a couple extra files besides the
extracted ISO content.

Post by Arnd Bergmann
As you still need device specific boot media,

Not really. If RK3399 supported picking up its proprietary blobs from
ESP (it's not there yet, but it does support reading them from special
GPT partitions), then you could create a single media that could install
Debian on both RK3399 and BCM2711.

And when I am suggesting that vendors provide UEFI support, ensuring
that the platform reading custom blobs from the ESP is also part of what
I have in mind, because if you do that, then I don't think the "But it
still needs device specific boot media" is receivable.

Of course, it may require distro maintainers to shift from the idea that
"people should only use DD when writing ISOHybrids". But then again,
considering that the whole point of UEFI was to make booting possible
from a simple media content extraction to a FAT32 partition (and even
for ARM there are simple ways [2] to work around the 4 GB FAT32
limitation, if your worry is that you may have a >4 GB file on your
ISO), this method should always have been something that image creators
need to consider, along with some awareness that DD mode is not always
the panacea that it's cracked up to be, as it can be very problematic
for Windows users [3].

Post by Arnd Bergmann
it does not solve the problem that
BBR tries to address by asking for UEFI support in the boot
firmware.

I disagree. The Pi 4 UEFI installation process demonstrates that we can
get pretty darn close to a universal Debian installation process from
vanilla ARM64 ISOs, where all you'd need to do to install on this or
that platform is drop a couple extra files into your ESP.

This also solves the rather important issue of Debian having to provide
custom images with non-free blobs, whereas you can now just ask users to
pick these files themselves and use a simple file copy operation to add
them.

Post by Arnd Bergmann

Post by Marcin Juszkiewicz

Post by Paul Wise
Storing boot firmware on the board seems to encourage various
sub-optimal things. Fully proprietary bootloaders. GPL violating
bootloaders. Bootloaders containing blobs. Bootloaders that are locked
down and cannot be replaced at all, especially on mobile. Forks of
random snapshots of existing bootloaders.
Personally I would go the other way; there should be a standard boot
protocol between the SoC boot ROM and later layers of the boot process
and vendors should stop shipping any boot firmware beyond the SoC boot
ROM, leaving it to OS distributors to package mainline bootloaders
that support that protocol.

Whatever allow SBC to have "unpack, connect, install, reboot, use".
Now on-boot-media bootloader can be whatever crap you can imagine.
And still requires maintaining all those /InstallDebianOn/RandomSBC wiki
pages, tools to generate installer images per board, tools to generate
bootable images per board etc.

Agreed, it is impossible to have board-agnostic boot media

If you're still thinking in terms of DD application of a vanilla Debian
ISO then yes.
But if you are thinking in terms of extracting ISO content onto FAT32 to
create UEFI bootable media, then I'd say you get fairly close to "board
agnostic boot media" if the SBC/SoC is designed to pick the additional
content it needs from a standard UEFI partition and the user just has to
file copy these missing elements.

Post by Arnd Bergmann
at the same
time as board-agnostic boot ROM unless you have some other place to
store board specific code and data, pretty much by definition. Whether
you store this in eMMC boot-partition, a spi-nor chip, or on-chip flash
doesn't matter too much, but most boards already have at least on of
those.

Just a note that I tried to informally see if the Raspberry Pi
Foundation would be considering bumping the size of the SPI flash on the
Pi 4 to accommodate the UEFI firmware (which has the downsize of always
being rather large. Even compressed, the Pi 4 one is larger than 2 MB),
but they didn't seem to be that receptive to the idea. If anything, they
went the other direction and removed on of the 2 SPI flash chips they
had on newer revisions.

Post by Arnd Bergmann
The missing bit that I see is getting board vendors to actually enable
UEFI support in their boot firmware, and storing (all of) it in the boot
partition rather than the eMMC data partition.

Or, like the Pi 4 does, be capable of storing and retrieving the UEFI
firmware and other proprietary blobs from the ESP...

Regards,

/Pete

[1] https://www.raspberrypi.org/forums/viewtopic.php?f=50&t=282839
[2] https://github.com/pbatard/uefi-ntfs
[3]
https://github.com/pbatard/rufus/wiki/FAQ#Why_doesnt_Rufus_recommend_DD_mode_over_ISO_mode_for_ISOHybrid_images_Surely_DD_is_better

[Andrew M.A. Cater]

Post by Pete Batard

Post by Arnd Bergmann
As far as I can tell (please let me know if this has changed), the only
way to get UEFI boot on Raspberry Pi is to load an EDK2 from
a special partition on a bootable drive (µSD or USB).

Not really.
You can load everything from a standard FAT32 EFI System Partition (ESP), as
least on the Raspberry Pi 4, which allows you to create media that is 100%
UEFI compliant from the get go (no need for special partitions or anything)
and also has the benefit of allowing you to install vanilla Debian using the
same media for the target and the source (without even having to do any
extra gymnastics for reboot), which can be a hugely beneficial thing on SBCs
where connectivity might be limited or where users may not have the luxury
of being able to afford 2 media.

Post by Arnd Bergmann
Create a GPT ESP (EFI System Partition) onto an USB, extract the
netinst.iso content there, add the latest Raspberry PI UEFI firmware
and proceed to a standard Debian networked installation. That's it!

Hi Pete

Post by Pete Batard
And by the way, this method of installing Debian from a single media by
leveraging the ESP is something that's also frequently used on x86 UEFI
based PCs, so there's really nothing "custom" about that method, apart from
the fact that you need to sprinkle a couple extra files besides the
extracted ISO content.

This didn't work for me yesterday on two occasions. What _did_ work was
creating the ESP partition and formatting it [using parted for the creation
of the partition and fdisk to mark it as fat32. I put in an ESP partition of
512M, which appears to be too small to hold the contents of the Bullseye
rc2. I then copied over the firmware and UEFI zip.

I used dd to put the iso onto a USB drive and used that to boot from.
Result: one successful install. [Only to discover that my RPi4 is one of the
very early ones and appears limited to 3GB even if I toggle the setting under
UEFI.

Post by Pete Batard
Of course, it may require distro maintainers to shift from the idea that
"people should only use DD when writing ISOHybrids". But then again,
considering that the whole point of UEFI was to make booting possible from a
simple media content extraction to a FAT32 partition (and even for ARM there
are simple ways [2] to work around the 4 GB FAT32 limitation, if your worry
is that you may have a >4 GB file on your ISO), this method should always
have been something that image creators need to consider, along with some
awareness that DD mode is not always the panacea that it's cracked up to be,
as it can be very problematic for Windows users [3].

See above - not a problem for me. In fact, the Raspberry Pi Foundations
imaging software appears to work fine to write an iso to an SD card or USB
and that or Rufus would be a better thing to advise.

Post by Pete Batard
Regards,
/Pete
[1] https://www.raspberrypi.org/forums/viewtopic.php?f=50&t=282839
[2] https://github.com/pbatard/uefi-ntfs
[3] https://github.com/pbatard/rufus/wiki/FAQ#Why_doesnt_Rufus_recommend_DD_mode_over_ISO_mode_for_ISOHybrid_images_Surely_DD_is_better

With thanks for everyone's hard work and best wishes as ever,

Andy C

[Pete Batard]

Hi Andrew,

Post by Andrew M.A. Cater

Post by Pete Batard
And by the way, this method of installing Debian from a single media by
leveraging the ESP is something that's also frequently used on x86 UEFI
based PCs, so there's really nothing "custom" about that method, apart from
the fact that you need to sprinkle a couple extra files besides the
extracted ISO content.

This didn't work for me yesterday on two occasions.

You're going to have to provide more details about what you did in these
2 attempts (preferably in the Debian thread on the Raspberry Pi forums
[1], since this is getting off topic).

People following the steps detailed at [1] don't appear to have much of
an issue, outside of the various problems that came from using an
unfinalized RC, and which should now be sorted. Also note that the guide
is for USB install only, since we are still missing an SD driver with
ACPI binding in mainline kernel, to work with UEFI. Some folks are still
working on it and it should naturally percolate to vanilla Debian media
once that's done just like networking and other things already have
(which is how you want things to happen). The guide makes it very clear
that it's USB only for now.

Post by Andrew M.A. Cater
What _did_ work was
creating the ESP partition and formatting it [using parted for the creation
of the partition and fdisk to mark it as fat32. I put in an ESP partition of
512M, which appears to be too small to hold the contents of the Bullseye
rc2.

So, for one thing you're using the full Bullseye rather than netinst.
That's one major deviation from the guide, though using full should work
too.

But the point of using netinst (or mini) is to avoid having to sacrifice
too much space for the ESP, since the install files are only used once.
In other words, if you need more than 512 MB, you're going to be wasting
a lot of space, in which case you're better of using 2 drives (one with
the installer, one for the target). But if you do that, then you have to
make sure that the UEFI firmware and Pi blobs get copied into the target
disk ESP before the first reboot, and this is something that is not
covered from the guide.

Post by Andrew M.A. Cater
I then copied over the firmware and UEFI zip.
I used dd to put the iso onto a USB drive and used that to boot from.
Result: one successful install. [Only to discover that my RPi4 is one of the
very early ones and appears limited to 3GB even if I toggle the setting under
UEFI.

I have a relatively early Pi 4 with 4 GB RAM, along with a more recent
one with 8 GB, and I am not aware of such a limitation. Do you have a
link to this?

We've had some issues getting the "more than 3 GB RAM" to stick when
changing the option in the UEFI firmware settings, but that should have
been fixed when using a recent version of it.

Post by Andrew M.A. Cater

Post by Pete Batard
Of course, it may require distro maintainers to shift from the idea that
"people should only use DD when writing ISOHybrids". But then again,
considering that the whole point of UEFI was to make booting possible from a
simple media content extraction to a FAT32 partition (and even for ARM there
are simple ways [2] to work around the 4 GB FAT32 limitation, if your worry
is that you may have a >4 GB file on your ISO), this method should always
have been something that image creators need to consider, along with some
awareness that DD mode is not always the panacea that it's cracked up to be,
as it can be very problematic for Windows users [3].

See above - not a problem for me. In fact, the Raspberry Pi Foundations
imaging software appears to work fine to write an iso to an SD card or USB
and that or Rufus would be a better thing to advise.

The guide at [1] also describes how one can use Rufus on Windows to
create the installation media (since Microsoft in their great wisdom
decided to make accessing ESP way more difficult than it should be on
recent versions of Windows).

Regards,

/Pete

Post by Andrew M.A. Cater

Post by Pete Batard
[1] https://www.raspberrypi.org/forums/viewtopic.php?f=50&t=282839

[LinAdmin]

Feel free to continue dreaming...

Post by Marcin Juszkiewicz
Let me bring the idea on this list that, maybe, it is
time*for vendors to start looking into *moving away from
using good old U-Boot on microsd or similar to provide
on-board SPI flash to store bootloader. So we could stop
distributing special installable images for platforms like
the Pi.

[Steve McIntyre]

Granted we also should have logged a bug for that issue as soon as we picked
it up, so we have to share part of the blame on this one. But the delays in
processing #985956, even though we tried to bring attention over and over
again to the fact that this very negatively affected one of the rare UEFI
installation of Bullseye on ARM64, on what has to be the current most
widespread system for that arch, was a bit concerning. Especially, it is not
the first time we've seen what I would qualify as extended delays in trying
to get showstopping UEFI patches looked into. As a matter of fact, I
eventually gave up trying to get the necessary Pi4 network patches backported
into Debian 10 (#950578), because even though we did submit a working patch
from the get go, it took 3 months (!) to actually get someone on the Debian
side to look into it, only to then tell us to just go re-do that patch, which
is not what I would qualify as a viable mode of operation.

So AFAICS what you're seeing is a severe lack of volunteer time to do
some things in Debian. I've been one of the more active Debian arm
porters in recent years, but I've moved on to other things and been
swamped with things like shim and grub security work in my "spare"
time for the last year or so. I'm also busy with a new job that (so
far) has very little to do with the Arm world, so I've not been able
to spend work time to pick up on some of these issues where prveiously
I might have done.

While this stuff is important for you and others, I'm afraid that
doesn't necessarily make it top priority for volunteers already
overloaded with other projects. That's not a lack of will to help,
it's simple logistics. :-( We need to get more people involved to
help, and that's often a struggle.

So, at the risk of being controversial, it doesn't seem to me like everybody
in the Debian world has been that willing to embrace UEFI. Or if they do,
then not everybody appears to have recognized the importance of being able to
provide Debian in UEFI mode on what has to account for the most popular ARM64
platform. And I could point to further evidence of this when I also brought
the idea on this list that, maybe, it was time for Debian to start looking
into moving away from using good old uboot or similar for distributing
special installable images for platforms like the Pi not that we have a full
blown UEFI, as this very idea seemed to irk a few people (which, to be fair,
may not have been directly affiliated with Debian).

As a Debian UEFI team person, I'm *really* happy that finally we have
an affordable platform that might work sensibly like this. As Marcin
says: if anything, our arm64 port has expected UEFI from the
get-go. There are still remnants of this assumption in d-i if you go
looking, where we don't work well enough on other platforms.

However, one of our biggest issues has been the ridiculously,
*appallingly* over-diverse Arm ecosystem with vendors continuing to
push special-snowflake SBCs that all need special consideration just
to do the simple basic things. It's batshit insane. On reflection,
that mess was one of the reasons why I changed jobs. It was causing
enough stress that I had to get away from it.

But regardless of this, my remark was aimed more at the topic at hand which
should be what feedback Debian may want to convey to the ARM community, and
especially the ARM platform manufacturers, and therefore goes beyond than
just the Raspberry Pi platform.
In short, I would reiterate that, in light of the headaches caused by all the
other approaches, and especially a requirement that still seems to boil down
to providing custom images (in part because of custom boot methods as well as
proprietary blobs, which I don't realistically see as going away) the
feedback Debian should provide to ARM vendors is that, if the latter want
better cooperation with Linux distros, then they should put the onus on
releasing a working UEFI firmware for their platform, especially as it's been
demonstrated that, even for an SBC as quirky and as ill-suited for it as the
Raspberry Pi (and that was part of the point of the whole exercise), it is
not that difficult to achieve.

+1000

[Paul Wise]

Post by Pete Batard

Post by Paul Wise
Will this be added to edk2-platforms?
https://github.com/tianocore/edk2-platforms/

https://github.com/tianocore/edk2-platforms/tree/master/Platform/RaspberryPi/RPi4
The Raspberry Pi 4 UEFI firmware has been an official EDK2
implementation for some time...

Hmm, then I don't understand the point of the rpi4-uefi.dev project
then. I guess it only exists because people don't want to compile
edk2-platforms themselves, the RPi Foundation doesn't offer
precompiled edk2-platforms, TianoCore doesn't offer precompiled
edk2-platforms and the RPi4 is the most popular platform supported by
edk2-platforms. Perhaps Debian should have edk2-platforms and
edk2-non-osi packages supporting all the devices to fill this gap.

[Pete Batard]

Post by Paul Wise

Post by Pete Batard
The Raspberry Pi 4 UEFI firmware has been an official EDK2
implementation for some time...

Hmm, then I don't understand the point of the rpi4-uefi.dev project
then. I guess it only exists because people don't want to compile
edk2-platforms themselves,

Yes. That is the point.

EDK2 did not have CI when we started integrating this work into it, and
the popularity of the platform made it obvious that people would want
pre-built version of the UEFI firmware that they can just drop on SD/USB
and run.

Even at this stage, it is not clear whether EDK2 is planning to dedicate
resources to have CI builds of platforms like the Raspberry Pi. So the
folks that have been responsible for the development and integration of
the UEFI firmware took matters in their own hands and created an
independent RPi4 GitHub repo, which essentially exists to provide builds
for end-users, as well as allow them to report issues with the firmware.

Regards,

/Pete

[Andrew M.A. Cater]

Post by Ralph Aichinger

Post by Paul Wise

Many criticisms of the RPi that were true 5 years ago no longer hold.

Some of them are still true; the weird GPU-starts-CPU SoC boot
process,

Some of them are hugely true: at the time when the first RPi was released, it
was released as ARM v6 with hardware floating point. Almost all Linux
distributions at that point had agreed on architectures <ARM v7 32 bit as
having software floating point and ARM v7 specs to be hardware floating
point.

Sounds small: but it's meant that Raspbian was always the odd one out, that
RPi .debs had to be essentially rebuilt from scratch and you couldn't
just move to stock Debian / Ubuntu or whatever or run software from one
on the other.. Raspberry Pi OS still preserves this v6-ness and essential
32 bit nature - not just for backward compatibility but because the Soc ine
very Pi Zero is still only ARM v6 and 32 bit.

This is also the reason why Raspberry Pi OS isn't particularly concerned with
64 bit and a 64 bit Raspberry Pi OS is in paermanent beta, though
you've been able to run 64 bit code on everything since some models of the
Raspberry Pi 2.

There have also been points when the Raspberry Pi Foundation didn't
talk to the wider ARM Linux community (and where the wider community couldn't
get information/offer advice/help) which hasn't helped the overall situation
or some people's feelings about Raspberry Pi in gneral

Strange booting process: a consequence of the original Broadcom SoC.
It would be _so_ nice if RPi people would just adopt UEFI from here on in
and produce firmware that would default to that.

The details of Raspberry Pi-ness and requests there are subtly different
to requests to ARM themselves. The fact that the RPi is very much dominant at
the moment is also because it's very much available while most of the other
ARM SBCs are on back order / not currently in production / waiting on chips.
If that situation continues,then the RPi will become the de facto solution
for most ARM stuff in the medium term.

Just my €0.02

Andy Cater

Post by Ralph Aichinger
Is this still true for the Raspberry Pi4 with UEFI?
https://github.com/pftf/RPi4
Even Debian Wiki says
https://wiki.debian.org/RaspberryPi
"All Raspberry Pi models before the 4 (1A, 1B, 1A+, 1B+, Zero, Zero W,
2, 3) boot from their GPU"
So it seems this is no longer true, and exactly what I said.

Post by Paul Wise
blobs required for the boot process,

If there are Blobs needed to bring up the RPi4 they are included
in above UEFi firmware (or the stuff used in other boot mechanisms).
I don't see how this is different from the "blobs" I load when I
boot my UEFi Asus mainboard.

Post by Paul Wise
some Linux kernelpatches are not in mainline etc

What patches do you mean? Patches that are in Debian kernels, but
not in Linus' upstream kernels? Or patches on top of what Debian
does? Because if you mean the latter, than I can assure you, that
the RPi4b runs fine without anything but a stock debian kernel,
both during installation (there the kernel from recent bullseye arm64 
netinst works just fine) as in actual use. The kernel I am currently
running is
ii linux-image-5.10.0-7-arm64 5.10.40-1 arm64 Linux
5.10 for 64-bit ARMv8 machines (signed)
directly from Debian archive.
And I stand by my statement: What was true 5 years ago (weird GPU boot,
patched proprietary kernel, architecture that falls right between
armel/armhf in Debian) is no longer true for kernels after 5.10
and with RPi4. The RPi4 works with stock Debian kernels (and supposedly
stock upstream kernels), runs straight arm64 just like other devices
and boots debian netinstall installation directly as long as you copy
it over onto an UEFi partiton with the UEFi EDK2 files included.

Post by Paul Wise
FreedomBox is a Debian blend, so a FreedomBox install *is* a Debian
install, there are no custom packages or other hacks. At least that
is how it is claimed to be, I haven't tried it.

I have not tried it either, but I am slightly worried about
small variations in the details. And I do not need the extra
software they install (yes, I could uninstall, of course).
My most recent Pi4 is used for taking backups with restic over
wireguard, so I more or less just need restic, wireguard, and
ssh (plus the usual commandline utilities). It would feel wrong
to install Freedombox, look what has to be uninstalled, look
for potential specific configurations etc.

Post by Paul Wise
I assume because of the proprietary software needed to boot the
RPi4,a lthough there is a WIP libre replacement that isn't yet in
Debian, see other comments in the thread about that.

With "WIP libre replacement" do you mean the tianocore/EDK2/UEFi port
here?
https://github.com/pftf/RPi4
Or something else? 
/ralph

[Ralph Aichinger]

Post by Andrew M.A. Cater
Some of them are hugely true: at the time when the first RPi was released, it
was released as ARM v6 with hardware floating point. Almost all Linux
distributions at that point had agreed on architectures <ARM v7 32 bit as
having software floating point and ARM v7 specs to be hardware floating
point.
Sounds small: but it's meant that Raspbian was always the odd one out, that
RPi .debs had to be essentially rebuilt from scratch and you couldn't
just move to stock Debian / Ubuntu or whatever or run software from one
on the other.. Raspberry Pi OS still preserves this v6-ness and essential
32 bit nature - not just for backward compatibility but because the Soc ine
very Pi Zero is still only ARM v6 and 32 bit.

Yes, this was a big thing, but five years ago! Or ten.
But I don't see how these old grudges are any more relevant
than if Acer preinstalls 32 or 64bit Windows on their
computers, or if ChromeOS on certain devices is 32 or 64 bit.

The Raspberry Pi 4 is completely capable of running plain vanilla
64bit aarch64/arm64 binaries. And that is what counts. Grudges
about historic decisions of the RP Foundation don't change that
at all.

Post by Andrew M.A. Cater
This is also the reason why Raspberry Pi OS isn't particularly concerned with
64 bit and a 64 bit Raspberry Pi OS is in paermanent beta, though
you've been able to run 64 bit code on everything since some models of the
Raspberry Pi 2.

They also have different goals from Debian. It is not surprising
that their solutions are different.

Post by Andrew M.A. Cater
There have also been points when the Raspberry Pi Foundation didn't
talk to the wider ARM Linux community (and where the wider community couldn't
get information/offer advice/help) which hasn't helped the overall situation
or some people's feelings about Raspberry Pi in gneral

That might be true, but then there is still the reality that the
RPi is probably the most successful ARM system outside of
mobile phones/tablets. It is what I can buy in the shops now,
at a very competitive (compared to amd64 and other arm boards)
price. It runs vanilla Debian without patches (unlike many
sunxi based devices with old old kernels). It is in the reach
of everybody (unlike the hardware that Amazon uses for their
arm based EC2 hosts, whatever that is, you cannot easily buy
it).

Post by Andrew M.A. Cater
Strange booting process: a consequence of the original Broadcom SoC.
It would be _so_ nice if RPi people would just adopt UEFI from here on in
and produce firmware that would default to that.

Who is "Raspberry Pi people"? There is a -- by now very stable --
UEFI support for the RPi4 (and 3? never tried that), so again
this criticism is not fair. If this happens with the blessing
and financing of the foundation (I have no idea) or not does
not make any difference.

Yes, the official foundation images do use something else, but
so do most hardware vendors (by preinstalling Windows). I don't
see how this is a valid criticism.

Post by Andrew M.A. Cater
The details of Raspberry Pi-ness and requests there are subtly different
to requests to ARM themselves. The fact that the RPi is very much dominant at
the moment is also because it's very much available while most of the other
ARM SBCs are on back order / not currently in production / waiting on chips.
If that situation continues,then the RPi will become the de facto solution
for most ARM stuff in the medium term.

It *is* the de facto solution for most people for the medium term,
at least until the dust around the Apple M1 systems settles (too
expensive to buy as an expensive toy with unknown support).

And I don't see how this is bad. It is not as if the RPi takes
away anybody's options to buy other Arm SBCs.

/ralph

[Paul Wise]

Post by Andrew M.A. Cater
Some of them are hugely true: at the time when the first RPi was released, it
was released as ARM v6 with hardware floating point. Almost all Linux
distributions at that point had agreed on architectures <ARM v7 32 bit as
having software floating point and ARM v7 specs to be hardware floating
point.

This is no longer relevant, since the arm64 and armhf capable RPi
boards are way more popular than the old boards now.

[Andrew M.A. Cater]

Post by Paul Wise

Post by Andrew M.A. Cater
Some of them are hugely true: at the time when the first RPi was released, it
was released as ARM v6 with hardware floating point. Almost all Linux
distributions at that point had agreed on architectures <ARM v7 32 bit as
having software floating point and ARM v7 specs to be hardware floating
point.

This is no longer relevant, since the arm64 and armhf capable RPi
boards are way more popular than the old boards now.
--
bye,
pabs
https://wiki.debian.org/PaulWise

Pi Zero is still ARM v6 :(

Andy C

[Jeffrey Walton]

Post by Andrew M.A. Cater

Post by Paul Wise
This is no longer relevant, since the arm64 and armhf capable RPi
boards are way more popular than the old boards now.
https://wiki.debian.org/PaulWise

Pi Zero is still ARM v6 :(

And the RPI-1's. I still have one to test Botan, Crypto++, GnuPG and
OpenSSL on ARMv6.

Jeff

[LinAdmin]

I see the main reason in an excellent ratio of price /
performance.

IMHO the Pi4 can be used as perfect low power replacement of
desktop system, home server, router etc. etc.

So if Debian wants to ignore that for beautiful ethical
reasons, I will switch to Ubuntu. And yes, I do know that
the famous respected decision makers here won't care.

The details of Raspberry Pi-ness and requests there are subtly differentto requests to ARM themselves. The fact that the RPi is very much dominant at
the moment is also because it's very much available while most of the other
ARM SBCs are on back order / not currently in production / waiting on chips.
If that situation continues,then the RPi will become the de facto solution
for most ARM stuff in the medium term.
Just my €0.02
Andy Cater

[LinAdmin]

You may critizise what ever you don't like, but RaPi
foundation does not care at all.
So what?

Post by Paul Wise

Many criticisms of the RPi that were true 5 years ago no longer hold.

Some of them are still true; the weird GPU-starts-CPU SoC boot
process, blobs required for the boot process, some Linux kernel
patches are not in mainline etc

[Peter Ehlert]

Post by Uwe Kleine-König
Hello,
next week there is a (virtual) meeting at ARM who invited some people
involved in Linux on ARM CPUs. One of the topics there is to tell them
Debian's needs and pain points.

where and how can I attend that meeting?

Post by Uwe Kleine-König
My current list (based on own experience and asking for feedback in
 - Fragmentation
   - Vendor kernels vs. mainline
     This got better in the past is my subjective impression, but it
     still hurts. Device tree made this a tad simpler, but it's not
     unusual to have vendor specific bindings.
   - early boot code
     U-Boot (or general: bootloader) is device specific and more often
     than not there is only a Vendor variant available.
     Also today there are more relevant components: ATF, UEFI/EDK2
   Vendors care at different intensities (and profit from external
   developers) Would Arm Base System Architecture (BSA) help? (This is
   only for AArch64 though, arm32 still relevant for us.)
   - Allwinner
   - Broadcom / RaspberryPi Foundation
   - Marvell
   - NXP
   - Odroid
   - Rockchip
   - some more for sure (which?)
 - Graphics
   Similar problematic, vendor blobs vs. OSS
Is there anything on your mind that is missing above and that you'd
like to be shared with ARM? Feel free to reply here or discuss in
#debian-arm. (I'm ukleinek there.)
Best regards
Uwe

[Uwe Kleine-König]

Hello Peter,

Post by Peter Ehlert

Post by Uwe Kleine-König
next week there is a (virtual) meeting at ARM who invited some people
involved in Linux on ARM CPUs. One of the topics there is to tell them
Debian's needs and pain points.

where and how can I attend that meeting?

You cannot. It's invite-only, sorry.

Best regards
Uwe

[Andrew M.A. Cater]

Post by Uwe Kleine-König
Hello,
next week there is a (virtual) meeting at ARM who invited some people
involved in Linux on ARM CPUs. One of the topics there is to tell them
Debian's needs and pain points.
My current list (based on own experience and asking for feedback in
- Fragmentation
- Vendor kernels vs. mainline
This got better in the past is my subjective impression, but it
still hurts. Device tree made this a tad simpler, but it's not
unusual to have vendor specific bindings.
- early boot code
U-Boot (or general: bootloader) is device specific and more often
than not there is only a Vendor variant available.
Also today there are more relevant components: ATF, UEFI/EDK2
Vendors care at different intensities (and profit from external
developers) Would Arm Base System Architecture (BSA) help? (This is
only for AArch64 though, arm32 still relevant for us.)
- Allwinner
- Broadcom / RaspberryPi Foundation
- Marvell
- NXP
- Odroid
- Rockchip
- some more for sure (which?)
- Graphics
Similar problematic, vendor blobs vs. OSS
Is there anything on your mind that is missing above and that you'd like to
be shared with ARM? Feel free to reply here or discuss in #debian-arm. (I'm
ukleinek there.)
Best regards
Uwe

Just get _someone_ to make a good quality 64 bit server which doesn't cost
the earth and works well with UEFI and relatively standard interfaces
and components.

AMD were doing this n years ago but the devices never got popular/cheap
enough for use. Marvell have the espressobin and macchiatobin - just
get something that looks like a performant mini-ATX / itx board and
can run forever at low power but in a standard form factor.

Andy C.

[Lennart Sorensen]

Post by Andrew M.A. Cater
Just get _someone_ to make a good quality 64 bit server which doesn't cost
the earth and works well with UEFI and relatively standard interfaces
and components.
AMD were doing this n years ago but the devices never got popular/cheap
enough for use. Marvell have the espressobin and macchiatobin - just
get something that looks like a performant mini-ATX / itx board and
can run forever at low power but in a standard form factor.

I remember seeing many arm servers announced. I don't recall really
seeing any you could buy unless you were a data center. If you were a
developer it seemed no one wanted to talk to you. And then they wondered
why no one was showing interest in their servers that no one could write
code for.

[Diederik de Haas]

Post by Andrew M.A. Cater
Just get _someone_ to make a good quality 64 bit server which doesn't cost
the earth and works well with UEFI and relatively standard interfaces
and components.

+1

Post by Andrew M.A. Cater
get something that looks like a performant mini-ATX / itx board and
can run forever at low power but in a standard form factor.

+10

Right now it seems that the manufacturer of your SBC must also have all needed
peripherals.
CIP: I have a Rock64, but it has a heating problem. But for that device
there's a tiny heatsink and an aluminum case, which acts/works as a 'giant'
heatsink. But there isn't an option for a case with a fan for example.

Possibly 'inspired' by my questions on irc:Pine64:#rock64, someone created a
3D printable top where one could attach a fan to the "Model B" Acrylic Open
Enclosure, which I have (bought from the Pine Store):
https://wiki.pine64.org/wiki/%22Model_B%22_Acrylic_Open_Enclosure

Absolutely wonderful :)
One 'tiny' problem though: I don't have a 3D printer.

So my request also in this context comes down to:
Can we have some standardizations?
Including in form-factors and (thus) casings.

Cheers,
Diederik

[Paul Wise]

Post by Uwe Kleine-König
Is there anything on your mind that is missing above and that you'd like
to be shared with ARM? Feel free to reply here or discuss in
#debian-arm. (I'm ukleinek there.)

Pretty sure just everything folks have been asking for for years can
be summarized in four words: libre, upstream, standards, supported.

Switch from proprietary drivers/firmware to open. Do everything
upstream not in forks. Standardise things across vendors. Vendors need
to support and continually update their hardware so we can continue to
buy and use it.

An example from #debian-arm last night: for SBCs a standard SoC boot
sequence and boot ROM would be nice, so we don't have to look up how
each SoC boots.

Oh, and since we are asking for ponies, make Apple let us boot free
software on iPhones ;)

[ibu ☉ radempa]

Post by Paul Wise
Pretty sure just everything folks have been asking for for years can
be summarized in four words: libre, upstream, standards, supported.

In the support section I'd add (although not specific to Debian) the
xpectation of timely support for attack mitigations (of the
Spectre/Meltdown kind), also for stable and maybe oldstable kernels.

BTW, can somebody add update [1], adding information for kernel 5.10?

[1]
https://wiki.debian.org/DebianSecurity/SpectreMeltdown#A64-bit_ARM_.28arm64.29

[Andrei POPESCU]

Post by Uwe Kleine-König
- Allwinner
- Broadcom / RaspberryPi Foundation
- Marvell
- NXP
- Odroid
- Rockchip
- some more for sure (which?)

There are some interesting Chromebooks (existing and announced), maybe
some SBCs as well, based on Mediatek SoCs[1].

Apple's M1 and future designs are also extremely interesting, e.g. a
Macbook Air fully supported by a mainline kernel would be *very*
interesting, despite the price[1].

Post by Uwe Kleine-König
- Graphics
Similar problematic, vendor blobs vs. OSS

Personal pain point: the PowerVR GX6250 in the Mediatek MT8173. But
since this is a meeting with ARM, having fully mainlined support for all
their GPUs would already be a great start (unsure what the current
status is).

Mainlined support for the various on-chip hardware decoders would also
be great (i.e. ffmpeg and friends in addition to Linux).

Post by Uwe Kleine-König
Is there anything on your mind that is missing above and that you'd like to
be shared with ARM? Feel free to reply here or discuss in #debian-arm. (I'm
ukleinek there.)

[1] writing this on an Acer Chromebook R13 running an ancient kernel
from archlinuxarm.org that would benefit from mainline support.

[2] I'm not aware of a similar fanless laptop

Kind regards,
Andrei

[Milan P. Stanić]

Post by Andrei POPESCU
There are some interesting Chromebooks (existing and announced), maybe
some SBCs as well, based on Mediatek SoCs[1].

I use chromebooks as my main workstations for about five years.
Started with samsung peach pi (arm32 exynos-5800) and one year later
bought acer R13 (arm64 mediatek mt8173), and again one year later bought
samsung one plus (arm64 rk3399).

Started to use them with debian and custom build kernels from chromeos
but later switched to mainline.
About three years ago I switched them to alpine linux (shameless plug:
I'm alpine developer but still follow debian infos) and they quite
usable though far from perfect. But again I had problems with intel
boxes also and always had to build kernels with patches and some quirks
also on them.

Post by Andrei POPESCU
Apple's M1 and future designs are also extremely interesting, e.g. a
Macbook Air fully supported by a mainline kernel would be *very*
interesting, despite the price[1].

My son bought one macbook pro M1 about eight months ago (he need it for
his job) and I was also tempted to buy one, but don't want to give so
much money to something which is to closed.

Post by Andrei POPESCU

Post by Uwe Kleine-König
- Graphics
Similar problematic, vendor blobs vs. OSS

Personal pain point: the PowerVR GX6250 in the Mediatek MT8173. But
since this is a meeting with ARM, having fully mainlined support for all
their GPUs would already be a great start (unsure what the current
status is).
Mainlined support for the various on-chip hardware decoders would also
be great (i.e. ffmpeg and friends in addition to Linux).

Post by Uwe Kleine-König
Is there anything on your mind that is missing above and that you'd like to
be shared with ARM? Feel free to reply here or discuss in #debian-arm. (I'm
ukleinek there.)

[1] writing this on an Acer Chromebook R13 running an ancient kernel
from archlinuxarm.org that would benefit from mainline support.

Also I'm writing this on 'Acer Chromebook R13' but with mainline kernel
5.12.10 with three patches I added. It is really good machine though it
is four years old. Only suspend-to-ram doesn't work with kernels 5.11
and 5.12 but it worked with previous kernels, 5.8 and 5.9 iirc.

Again shameless plug: I wrote small note and script to install alpine
linux on R13, here
https://arvanta.net/alpine/install-alpine-on-acer-r13-chromebook/

Post by Andrei POPESCU
[2] I'm not aware of a similar fanless laptop

Yes, quit quiet.

[Andrei POPESCU]

Post by Milan P. Stanić

Post by Andrei POPESCU
[1] writing this on an Acer Chromebook R13 running an ancient kernel
from archlinuxarm.org that would benefit from mainline support.

Also I'm writing this on 'Acer Chromebook R13' but with mainline kernel
5.12.10 with three patches I added.

Are those patches published somewhere (and what are they for)?

Post by Milan P. Stanić
It is really good machine though it
is four years old. Only suspend-to-ram doesn't work with kernels 5.11
and 5.12 but it worked with previous kernels, 5.8 and 5.9 iirc.
Again shameless plug: I wrote small note and script to install alpine
linux on R13, here
https://arvanta.net/alpine/install-alpine-on-acer-r13-chromebook/

As far as I can tell that script must be run from within an already
existing Alpine Linux install. Am I guessing correctly that the package
linux-elm contains the kernel image? I'd rather try and extract the
kernel + modules only as I have no intention to move away from Debian.

A list of options that must be enabled in the kernel for the R13, as
well instructions for building the image to be flashed to the kernel
partition would also be very helpful.

With these we might be able to get full support for the R13 in the
Debian kernel and flash-kernel.

Kind regards,
Andrei

[Milan P. Stanić]

Post by Andrei POPESCU

Post by Milan P. Stanić

Post by Andrei POPESCU
[1] writing this on an Acer Chromebook R13 running an ancient kernel
from archlinuxarm.org that would benefit from mainline support.

Also I'm writing this on 'Acer Chromebook R13' but with mainline kernel
5.12.10 with three patches I added.

Are those patches published somewhere (and what are they for)?

Yes, here:
https://git.alpinelinux.org/aports/tree/testing/linux-elm

one is to fix mmc devices order to not be random.
one to 'cut' down external mmc frequency.
and one to fix spi nor frequency.

Post by Andrei POPESCU

Post by Milan P. Stanić
It is really good machine though it
is four years old. Only suspend-to-ram doesn't work with kernels 5.11
and 5.12 but it worked with previous kernels, 5.8 and 5.9 iirc.
Again shameless plug: I wrote small note and script to install alpine
linux on R13, here
https://arvanta.net/alpine/install-alpine-on-acer-r13-chromebook/

As far as I can tell that script must be run from within an already
existing Alpine Linux install.

Yes, but if using apk-tools-static pkg (alpine package manager) it can
be run from any linux distribution.

Post by Andrei POPESCU
Am I guessing correctly that the package > linux-elm contains the
kernel image? I'd rather try and extract the
kernel + modules only as I have no intention to move away from Debian.

Right, you can download linux-elm pkg from:
https://dl-cdn.alpinelinux.org/alpine/edge/testing/aarch64/linux-elm-5.12.10-r0.apk
and untar it. You will have modules and vmlinux.kpart image in /boot dir
which you can flash (dd) to chromebook kernel partition.

note: I didn't intended to tell that you or anyone should move from
debian, just wanted to help by pointing to that what I did to get R13
work with mainline kernels.

Post by Andrei POPESCU
A list of options that must be enabled in the kernel for the R13, as
well instructions for building the image to be flashed to the kernel
partition would also be very helpful.

In /boot you can find kernel config file which you can use as a base for
your own kernels.

Post by Andrei POPESCU
With these we might be able to get full support for the R13 in the
Debian kernel and flash-kernel.

You can look at
https://git.alpinelinux.org/aports/tree/testing/linux-elm/APKBUILD which
is alpine build recipe and is written in posix shell (ash) and look how
it is built.

Some of this ideas are 'stolen' from Arch Linux alarm

[Uwe Kleine-König]

Post by Milan P. Stanić

Post by Andrei POPESCU
Are those patches published somewhere (and what are they for)?

https://git.alpinelinux.org/aports/tree/testing/linux-elm
one is to fix mmc devices order to not be random.

That should better be done in the machine.dts. See for example
https://git.kernel.org/linus/5dcbe7e3862dfc89d219f37a9ed5e53944fa13c2

Post by Milan P. Stanić
one to 'cut' down external mmc frequency.

This should probably sent to mainline. With a proper change log this
should be easy.

Post by Milan P. Stanić
and one to fix spi nor frequency.

This increases spi nor frequency. The only benefit should be to speed up
reading/writing, so it's not necessary to make the machine work
correctly. If you mention the actual used part this should also be easy
to bring into mainline.

Best regards
Uwe

[Milan P. Stanić]

Post by Uwe Kleine-König

Post by Milan P. Stanić

Post by Andrei POPESCU
Are those patches published somewhere (and what are they for)?

https://git.alpinelinux.org/aports/tree/testing/linux-elm
one is to fix mmc devices order to not be random.

That should better be done in the machine.dts. See for example
https://git.kernel.org/linus/5dcbe7e3862dfc89d219f37a9ed5e53944fa13c2

Yes, I've seen this patch in my local git clone and even thought to post
this I made but not sure where to post it.

Post by Uwe Kleine-König

Post by Milan P. Stanić
one to 'cut' down external mmc frequency.

This should probably sent to mainline. With a proper change log this should
be easy.

Actually I looked in Arch Linux Alarm (and refactored it somewhat) when made
this patch and don't want to 'stole' it in a hope that original author
will post it.

Post by Uwe Kleine-König

Post by Milan P. Stanić
and one to fix spi nor frequency.

This increases spi nor frequency. The only benefit should be to speed up
reading/writing, so it's not necessary to make the machine work correctly.
If you mention the actual used part this should also be easy to bring into
mainline.

TBH I even forgot for what reason I added this one, so not sure even if
it is still needed.

[Arnd Bergmann]

Post by Uwe Kleine-König
Is there anything on your mind that is missing above and that you'd like
to be shared with ARM? Feel free to reply here or discuss in
#debian-arm. (I'm ukleinek there.)

I'll be at the meeting as well, and one point I would want to bring up
is support for 32-bit binaries on 64-bit kernels. This has come up a few times
on #debian-arm, and the problem is a mismatch between which obsolete
instructions are emulated depending on the kernel: 64-bit kernels emulate
all the instructions that were removed in v8 (setend, cp15 barriers, swp),
while 32-bit kernels emulate instructions that were already needed in
v7 (swp, mrc tlsreg, unaligned ldm/stm/ldrd/strd). This means that some
binaries that ran fine on an ARMv7 processor may break when running
on an ARMv8 processor with a 32-bit kernel, while other binaries may
break on the same hardware on a 64-bit kernel.

This situation is bad for Debian as there are mixed messages regarding
what users should actually run on ARMv8 hardware and what should
be tested. My hope is that we can find an agreement regarding which
of the emulation code we actually want on v8 processors and then make
sure that every binary that can run on a 32-bit kernel can also run on
a 64-bit kernel (but not necessarily the other way round). This could
mean adding features to arch/arm64/ that have previously been rejected,
or artificially crippling the emulation code in arch/arm/ when running on
v8 to force user space applications to get fixed.

Side note: yes, 32-bit user space code is still important to run for
a lot of users, especially in low-memory configurations, but support
for 32-bit kernels is unavailable on most newer CPU cores and
somewhat lacking even on older cores (missing errata workarounds
and certain features of 64-bit kernels).

Arnd

[Gene Heskett]

On Thu, Jun 10, 2021 at 4:43 PM Uwe Kleine-König

Post by Uwe Kleine-König
Is there anything on your mind that is missing above and that you'd
like to be shared with ARM? Feel free to reply here or discuss in
#debian-arm. (I'm ukleinek there.)

I'll be at the meeting as well, and one point I would want to bring up
is support for 32-bit binaries on 64-bit kernels. This has come up a
few times on #debian-arm, and the problem is a mismatch between which
obsolete instructions are emulated depending on the kernel: 64-bit
kernels emulate all the instructions that were removed in v8 (setend,
cp15 barriers, swp), while 32-bit kernels emulate instructions that
were already needed in v7 (swp, mrc tlsreg, unaligned
ldm/stm/ldrd/strd). This means that some binaries that ran fine on an
ARMv7 processor may break when running on an ARMv8 processor with a
32-bit kernel, while other binaries may break on the same hardware on
a 64-bit kernel.
This situation is bad for Debian as there are mixed messages regarding
what users should actually run on ARMv8 hardware and what should
be tested. My hope is that we can find an agreement regarding which
of the emulation code we actually want on v8 processors and then make
sure that every binary that can run on a 32-bit kernel can also run on
a 64-bit kernel (but not necessarily the other way round). This could
mean adding features to arch/arm64/ that have previously been
rejected, or artificially crippling the emulation code in arch/arm/
when running on v8 to force user space applications to get fixed.
Side note: yes, 32-bit user space code is still important to run for
a lot of users, especially in low-memory configurations, but support
for 32-bit kernels is unavailable on most newer CPU cores and
somewhat lacking even on older cores (missing errata workarounds
and certain features of 64-bit kernels).

And while discussing that, emphasize that some of us are running 32 bit
armhf because the stack frame is smaller when switching context, giving
better, measurable latency responses which are extremely important if
driving a stepper motor by software generated steps. It is not uncommon
to see an 80% speed loss when driving steppers by software steppers
compared to driving them with programmable fpga hardware. A 10
microsecond wobble in the step timing can cost us 50% of the motors
torque. 100 microseconds brings the motor to a complete stop but
bouncing back and forth in the magnetic grip, and if the next step
arrives when the motor is backing up, it can't accellerate to get back
in position, the part being made is ruined, and the motor is stalled,
losing track of its home position so the whole machine has to be
rehomed. That costs money in time wasted recovering, in raw materiel
ruined, and potentially broken and expensive tooling.

Going to pure 64 bit systems effectively means you are locking a large
percentage of the cnc manufacturing on this planet out in the parking
lot because we can't tolerate the loss of performance in real time that
bigger stack frame causes unless we spend an additional $200+ on
hardware interfaces to get that performance back. We may as well use a
bigger, uses 20x the power but its cheaper pc. I'm doing it both ways,
but my shop isn't commercial, its hobbiest. I don't have to make xx
dollars per hour for a living.

One old farts $0.02.

Arnd

Cheers, Gene Heskett

[Diederik de Haas]

Post by Uwe Kleine-König
- Fragmentation
- Vendor kernels vs. mainline
This got better in the past is my subjective impression, but it
still hurts. Device tree made this a tad simpler, but it's not
unusual to have vendor specific bindings.

Mainline would be best, but if (initially) a vendor kernel 'needs' to be made,
please don't use some random kernel version, but base it off an Extended Long
Term Support kernel. IIUC 5.10 is intended to be an ELTS kernel.

It also has the 'side' benefit that it (usually) aligns with a Debian Stable
release, which means various tooling should also easily work. And it should
also mean that it's 'easy' to get it working on other distros as well.

Cheers,
Diederik

[Uwe Kleine-König]

Hello Diederik

Post by Diederik de Haas

Post by Uwe Kleine-König
- Fragmentation
- Vendor kernels vs. mainline
This got better in the past is my subjective impression, but it
still hurts. Device tree made this a tad simpler, but it's not
unusual to have vendor specific bindings.

Mainline would be best, but if (initially) a vendor kernel 'needs' to be made,
please don't use some random kernel version, but base it off an Extended Long
Term Support kernel. IIUC 5.10 is intended to be an ELTS kernel.
It also has the 'side' benefit that it (usually) aligns with a Debian Stable
release, which means various tooling should also easily work. And it should
also mean that it's 'easy' to get it working on other distros as well.

In my bubble most vendor kernel already do this. They align however not
to an ELTS kernel, but to the Android universe. That's at least the case
for NXP which is quite dominant in my bubble.

Best regards
Uwe

[Diederik de Haas]

Hi Uwe,

Post by Uwe Kleine-König
In my bubble most vendor kernel already do this. They align however not
to an ELTS kernel, but to the Android universe. That's at least the case
for NXP which is quite dominant in my bubble.

Lucky you :)
I'm not very familiar with the Android universe and I see it separate from the
rest of the Linux universe. (my android phone (SGS7) runs 3.18 ...)
My router from 2017 has an ARM cpu, but uses kernel 3.4.103 ... _sigh_
Maybe/hopefully this is just anecdotal and dated and things are aligned/
converged much more now.

But generally speaking, if only 1 or a few kernels would be used by ARM
vendors, then it's easier to combine efforts and help each other, rather then
having everyone on their own island.
An ELTS kernel seems well suited for that.

Cheers,
Diederik

[Uwe Kleine-König]

Hello,

Post by Diederik de Haas

Post by Uwe Kleine-König
In my bubble most vendor kernel already do this. They align however not
to an ELTS kernel, but to the Android universe. That's at least the case
for NXP which is quite dominant in my bubble.

Lucky you :)
I'm not very familiar with the Android universe and I see it separate from the
rest of the Linux universe. (my android phone (SGS7) runs 3.18 ...)

I also have a Galaxy S7 running 3.18.91 ...

Post by Diederik de Haas
My router from 2017 has an ARM cpu, but uses kernel 3.4.103 ... _sigh_

... and my front router has 2.6.39.4, you can buy this one still today.

Post by Diederik de Haas
Maybe/hopefully this is just anecdotal and dated and things are aligned/
converged much more now.
But generally speaking, if only 1 or a few kernels would be used by ARM
vendors, then it's easier to combine efforts and help each other, rather then
having everyone on their own island.
An ELTS kernel seems well suited for that.

IMHO the version to pick is always the current tip of the development
tree. Assuming the vendors get their changes into mainline users are
free to use any later kernel. That is more useful than restricting to an
ELTS kernel. With an ELTS kernel the users wail that it is too old
and/or the vendors have the pain to update their patch stacks to the
next ELTS version which makes handling at their hand more painful, not
only because several (maybe even paying) customers have machines in the
field and are unwilling to do a major kernel bump for these machines and
so they have to support several versions anyhow.

Best regards
Uwe

[Diederik de Haas]

Hi!

Post by Uwe Kleine-König
IMHO the version to pick is always the current tip of the development
tree.

I understand that from a (kernel) developer's POV.

Post by Uwe Kleine-König
Assuming the vendors get their changes into mainline users are
free to use any later kernel.

That may be true if *everything* is FLOSS.
But my router (Asus BRT-AC828; 3.4.103) has very likely also binary firmware
(for Wifi AC), which makes me doubt I'll be able to do that.
I actually do have the source code for this router('s latest firmware).

While I am a technical user and I should be able to update busybox/samba f.e.
to a later version, writing kernel code to update to (f.e.) the 4.4 kernel [1]
(ah, I meant *Super* Long Term Support kernel), while keeping my Wifi working,
is WAY out of my league.
(A 4.x kernel should make WireGuard possible IIUC)

Post by Uwe Kleine-König
That is more useful than restricting to an ELTS kernel.
With an ELTS kernel the users wail that it is too old
and/or the vendors have the pain to update their patch stacks to the
next ELTS version

I'm looking at this from a *user*'s POV (with some technical knowledge).
I can be wrong, but it appears to me that what happens most often is that a
product is released with a specific kernel and will never be updated again (by
the manufacturer). Assuming they indeed started with the tip of the tree,
that'll result in a random kernel version being supplied with that product.

It would be awesome if the manufacturer would then update it to an ELTS/SLTS
version, but I don't expect to see that happen in my lifetime ;) [2]

So the *user* is stuck with a random kernel version.
If the user is 'stuck' with a ELTS/SLTS kernel version, just like a whole
bunch of other (technical) people ('on the internet'), then the chances of
collaborating with them to prolong the lifetime and security of their
product(s) go way up.

There are likely differences between product groups and therefor lifetime, but
I had my router in mind with my replies.

Cheers,
Diederik

[1] https://lwn.net/Articles/749530/
[2] I totally understand why they wouldn't/won't do it. It's a major (support)
risk and they much rather sell me a new product then let me prolong the
lifespan of an already bought product.

[Jeffrey Walton]

Post by Diederik de Haas
...
[1] https://lwn.net/Articles/749530/

That's an interesting project. Thanks for that link.

Insecure infrastructure (and other gadgets, like home routers) is such
a problem I hope the CIP project can get funding from places like the
National Science Foundation or Homeland Security (in the US).

Maybe the GPL license needs an update that addresses the abandonware
problem. The update clause requires vendors on top of the base system
to provide updates for the life of the device. Something like, "GPL
plus the update clause".

The update clause will ensure vendors fix their bugs and send them
upstream. The update clause won't affect the kernel because it is
being updated.

Jeff

[Richard Owlett]

Post by Jeffrey Walton

Post by Diederik de Haas
...
[1] https://lwn.net/Articles/749530/

The article's title is "Super long-term kernel support".

Post by Jeffrey Walton
That's an interesting project. Thanks for that link.
Insecure infrastructure (and other gadgets, like home routers) is such
a problem I hope the CIP project can get funding from places like the
National Science Foundation or Homeland Security (in the US).
Maybe the GPL license needs an update that addresses the abandonware
problem. The update clause requires vendors on top of the base system
to provide updates for the life of the device. Something like, "GPL
plus the update clause".
The update clause will ensure vendors fix their bugs and send them
upstream. The update clause won't affect the kernel because it is
being updated.
Jeff

[Adrian Bunk]

Post by Jeffrey Walton
...
Maybe the GPL license needs an update that addresses the abandonware
problem. The update clause requires vendors on top of the base system
to provide updates for the life of the device. Something like, "GPL
plus the update clause".
...

The GPL license does already address the abandonware problem,
by enabling you to fix the software yourself.

Abandonware is not limited to security updates,
it also includes topics like battery replacement.

Any solution to the abandonware problem for consumer products has to be
political, you need a law mandating for how many years repairs and
security updates have to be available.

Post by Jeffrey Walton
Jeff

cu
Adrian

[John Paul Adrian Glaubitz]

Post by Adrian Bunk
Any solution to the abandonware problem for consumer products has to be
political, you need a law mandating for how many years repairs and
security updates have to be available.

vote with your wallet

This is actually the only solution you have. You cannot force any manufacturer
to provide support, service parts and updates for many years if you buy a simple
consumer device.

There are companies that offer products with longer support times, but those are
naturally more expensive because providing support over such a long time costs
a lot of money to maintain.

My old iPhone 5s that was released in 2013 is still receiving security updates after
all these years. So there are actually companies offering that service. But again,
an iPhone is more expensive than a comparable Android device.

Adrian

[Diederik de Haas]

Post by John Paul Adrian Glaubitz
This is actually the only solution you have.

My response was based on disagreeing with "the only solution", which Adrian
Bunk made too. Consequently, I also disagree with your statement.

As almost all companies only seem to care about money and laws are considered
obstacles to work around*, I do think 'voting with your wallet' is more
effective.

*) Mobile phone are now 'required' to receive 2 year (security) support, which
means that there will be *one* software/OS updated provided in those 2 years.
Technically complying with the law, but surely not with its spirit.

Post by John Paul Adrian Glaubitz
My old iPhone 5s ... is still receiving security updates

Apple does good in that regard. 'Not so good' in others.

But financially rewarding a walled garden is something I will never do.
It goes against any freedom-loving bone in my body.

Diederik

[Diederik de Haas]

Post by Adrian Bunk
Any solution to the abandonware problem for consumer products has to be
political, you need a law mandating for how many years repairs and
security updates have to be available.

There is another type of solution that you can do right now:

vote with your wallet