Discussion:
Bug#1029843: live-boot: Devices Requiring Firmware: multiple requested files in single line overlapping / special characters
(too old to reply)
James Addison
2023-05-03 16:40:01 UTC
Permalink
After editing and rebuilding the Device Tree (DTS) files, and
deploying those changes to the system, I can confirm that adjusting
the 'model' field value in there has no effect on the requested fw
filename.
Did those modifications stay in place once you switched to Device Tree
in your bootloader configuration? Just wondering whether you tested two
cumulative changes (DTS tweaks + switch from ACPI), or independent ones
(DTS tweaks, then switch from ACPI but using pristine DTB files).
That was tested cumulatively, yep - applied the DTS tweaks, found no
change, and then updated the UEFI setting from ACPI to Devicetree,
resulting in the expected fw requests.

Since then I've reverted the DTS tweaks, and the correct behaviour continues.

After that I reverted the UEFI setting back to ACPI briefly. I forget
what I was checking, but the problem reappeared, and now I'm back to
the expected behaviour (no spaces in the fw filenames) with the
Devicetree setting.
I don't have any Pi 400 and haven't been following what the stock
configuration is (and sorry I didn't read the whole backstory)… if EDF2
UEFI comes by default, or is recommended, or fixes/works around bugs,
and in the end is expected to be relevant and widely used, and if ACPI
is indeed some kind of default setup, it would be best if we were to
support that.
I'd defer to people with more familiarity of the ecosystem on these
kind of questions, although am also keen to learn more.

Some thoughts:

* Maybe there's a bug to report in the upstream Linux brcmfmac
driver here; are there better alternatives than DMI that it could use
to determine fallback filenames? (again, I'm not sure, but can follow
up on that)
* Perhaps Devicetree is a better default in EDK2 for ARM systems?
(that wouldn't solve the root cause, though)
* Alternatively, if EDK2/RPI4-UEFI became Debian-packaged (excluding
the brcm firmware, because that's already in raspi-firmware), _and_
could be installed on the ESP partition by d-i (it's beyond my
experience to say whether that's a good idea...), then perhaps it
could be preconfigured (or d-i could request) Devicetree at
install-time.

If & when attempting this again - it could be a while, this took some
energy - then I would probably experiment with u-boot as a comparison.
I suppose that would mean either having the relevant files/symlinks in
the firmware package *and* d-i support for it (hw-detect limitations…);
or have some on-the-fly conversion in the Linux module so that it ends
up requesting files that are actually in the firmware package, and that
d-i can work with, without requiring any changes?
At the moment, I think that fixing this in the brcmfmac driver would
resolve the problem in a bootloader-agnostic way, so that seems worth
exploring.

Symlinks seem like they'd be a reasonable short-term workaround in our
packaging - but likely maintainer discretion on that one, as usual (if
that were me, it would help to know that it would be a temporary
measure while other problems were being resolved).

(sorry for what I think may be an overly-large reply list here - I'd
prefer that than to have anyone miss some hopefully-relevant details)
James Addison
2023-05-03 22:30:01 UTC
Permalink
Package: src:linux
Followup-For: Bug #1029843
X-Debbugs-Cc: ***@akeo.ie, ***@debian.org, ***@cknow.org, debian-***@lists.debian.org, ***@bugs.debian.org, ***@bugs.debian.org, ***@bugs.debian.org, debian-***@lists.debian.org
Control: retitle -1 brcmfmac: requested firmware filename inconsistent with linux-firmware.git on non-devicetree systems

Thanks, Pete.

I added a note[1] on the rpi4-uefi.dev GitHub repository, and from one of your
fellow contributors' responses, it seems that in fact the filename-with-spaces
format _is_ referenced from linux-firmware.git, in a 'WHENCE' file that is
used to create symlinks (I hadn't been aware of that previously).

As a result: I feel that maybe this bugreport is not valid.


I suppose that some of the confusion stemmed from the fact that a single binary
of a kernel module in combination with a single physical hardware device probed
different firmware filenames at runtime depending on the context (ACPI vs
Devicetree, in this case).

(it's code, so yep, I get that it's technically _possible_ for that to happen,
and perhaps it's useful to workaround limitations of existing standards, but
it's not clear to me whether that's necessary here)
Note that, in case you think there may be something that we can improve
in the SMBIOS data reported by the UEFI firmware (which is currently
generated from the source code at [1], with the full output from a
Raspberry Pi 4, from UEFI Shell's smbiosview command at [2]) we can look
into updating the UEFI firmware to alter the data we output.
Thank you - I'll take a look at those to learn more.

[1] - https://github.com/pftf/RPi4/issues/76#issuecomment-1533295773
James Addison
2023-05-04 13:30:01 UTC
Permalink
[ replying with some re-ordering ]
Obviously, with the idea of not having ARM based device that are
constrained to a single OS (be it Windows, Linux, BSD or something> else), and considering that Windows and Device Tree don't work together,
you want to go with a mode of operation that isn't Linux specific,
which, even if ACPI has its drawbacks, pretty much forces the use of
ACPI over Device Tree. Else, you have Linux going into the exact
Microsoft strong-arm tactics that it should strive to avoid...
Yep, and for those situations, that's a point in favour of the third
"System Table Selection" value that I failed to mention:
"ACPI+Devicetree".

I'm cautious about recommending it, given an understanding that
enabling ACPI could increase the amount of non-free code (ACPI Machine
Language, in particular) that may run on the system as a result.
Perhaps there could be counterbalancing functionality benefits and/or
energy-usage savings... even then I'd recommend proceeding cautiously.

I'm not sure I'm qualified to say much about Debian's compatibility
with other operating systems on the same machine, other than to
mention that I do think it's highly compatible and that that's
something that maintainers, developers and users care about.
Post by James Addison
* Perhaps Devicetree is a better default in EDK2 for ARM systems?
(that wouldn't solve the root cause, though)
Please note that the reason why the Raspberry Pi UEFI firmware defaults
to ACPI is so that this ARM device follows the relatively new ARM SBBR
standard [3], which we (hopefully) expect more and more ARM64 based
device to follow.
Slightly off-topic: do you know of cases where ACPI has helped a
vendor to adapt to shifting operating system interfaces or achieve
significant energy-usage savings? I think that understanding some of
those could help to begin to address gaps that Debian/Linux/other
components have.

(and to mention why I ask.. I've been reading some of the history[1],
definitions[2], rationale[3], and state-of-support[4] around
DeviceTree and ACPI in Linux, including in relation to ARM servers.
it looks like I have a decade-or-so of history to catch up on there)

[1] - https://lore.kernel.org/linux-arm-kernel/CAOesGMjKeRb=fFJM0MabDihbEiCGM4EqW9D5i_6-***@mail.gmail.com/

[2] - https://elinux.org/Device_Tree_What_It_Is

[3] - https://www.secretlab.ca/archives/151

[4] - https://www.kernel.org/doc/html/v6.3/arm64/arm-acpi.html
Pete Batard
2023-05-05 12:10:01 UTC
Permalink
Hi James,

Since we're getting off-topic, and I don't think there's much of this
reply that's going to be relevant to it, I dropped the CC to bug 1035392.
Post by James Addison
Yep, and for those situations, that's a point in favour of the third
"ACPI+Devicetree".
Indeed, the firmware provides that that option as well.
Post by James Addison
I'm cautious about recommending it, given an understanding that
enabling ACPI could increase the amount of non-free code (ACPI Machine
Language, in particular) that may run on the system as a result.
I think with current CPUs (especially on the x86 side with Management
Engines as well as proprietary blobs), we're alas way past the point of
being able to prevent non-free code from running.

The Raspberry Pi SoC has also some very non-free code running that
executes prior to the running of the UEFI firmware and also in parallel
to the UEFI firmware and OS. There's basically a Management Engine
running on the GPU, which, among other things, provides a mailbox that
the UEFI firmware uses to retrieve or set hardware configuration data.

On the other hand in this specific case, though I understand you're
speaking in a more general manner about ACPI usage, the whole ACPI blob
generation comes entirely from open source code.
Post by James Addison
Perhaps there could be counterbalancing functionality benefits and/or
energy-usage savings... even then I'd recommend proceeding cautiously.
As far as I'm concerned, the main reason I wouldn't advocate ACPI +
Device Tree is that it can create user confusion as to what is really
being used behind the scenes. A bit like when PC end-users enable Legacy
boot in their UEFI settings and end up installing their OS in Legacy
mode on a UEFI capable system, then find themselves in a situation where
they want to use UEFI features but can't.

Also, again, since part of our goal has been to promote the emerging
SBBR standard (because we think it should ultimately help making
installation of various OSes on par with what is the case for x86 based
PCs, where you can pretty much just create a universal boot media as
well as have the OS install and use unified boot loaders, rather than
force users to concern themselves about the low-level system specifics
of their system, and play with yet another custom version of u-boot),
and SBBR made the choice to go with ACPI only rather than
ACPI+DeviceTree, we decided to propose ACPI+DeviceTree as a means not to
restrict user choice for people who want to use both, but knowing that
it's not something we really want to officially support.
Post by James Addison
Slightly off-topic: do you know of cases where ACPI has helped a
vendor to adapt to shifting operating system interfaces or achieve
significant energy-usage savings?
I'm afraid I'm ill placed to discuss ACPI specifics, as, despite having
contributed one or two patches here and there to the Raspberry Pi UEFI
firmware ACPI bindings, my knowledge of ACPI is very limited.

With regards to adapting to OS interfaces my guess, even if it seems to
be at odds with the Linux kernel efforts of the past few years, is that
even with its limitations and constraints it was probably better to go
with a well established and relatively well supported implementation
that can effectively be shared with multiple OSes, rather than ask
hardware manufacturers to juggle with multiple implementations when
providing a functional implementation that complies with a specific
standard, especially a new one. So I would say that it was probably
decided that, since vendors were expected not to go with a standard that
would force them to "duplicate" some work, the burden of having to
support ACPI and only ACPI would be shifted to the OS folks, even if it
went against the direction that Linux had already been taking at that
stage...

But again, this is pure speculation on my part, and I can't tell if
ultimately picking up ACPI only will actually help vendors to go with
SBBR and, hopefully, better adapt to shifting OS interfaces, since the
idea is that OSes should also strive to follow SBBR... even if that
means, in the case of Linux, having to support ACPI alongside Device
Tree. No idea about energy savings though.
Post by James Addison
I think that understanding some of
those could help to begin to address gaps that Debian/Linux/other
components have.
I suspect that if you post your question to the edk2-discuss mailing
list [1], which is frequented by the same developers as edk2-devel and
therefore should be seen by people who are very knowledgeable about
ACPI, you might be able to get insightful answers to your question.

Regards,

/Pete

[1] https://edk2.groups.io/g/discuss
James Addison
2023-05-08 11:40:01 UTC
Permalink
Post by Pete Batard
Post by James Addison
Yep, and for those situations, that's a point in favour of the third
"ACPI+Devicetree".
Indeed, the firmware provides that that option as well.
Post by James Addison
I'm cautious about recommending it, given an understanding that
enabling ACPI could increase the amount of non-free code (ACPI Machine
Language, in particular) that may run on the system as a result.
I think with current CPUs (especially on the x86 side with Management
Engines as well as proprietary blobs), we're alas way past the point of
being able to prevent non-free code from running.
Somewhat agree, yep - although I think that in this case, there should
be various paths available (u-boot, EDK2, ACPI/DT), and if possible
I'd like to understand what is the approach that provides the most
compatibility and free software support with the fewest moving parts.
To me, the reliability and human-time cost savings from simpler, more
open and straightforward systems outweigh many other factors,
especially over the long term.

That's partly the reason I've been wondering about the power
consumption and operating-system-compatibility questions: as I
understand it, those were key reasons that server hardware vendors had
a preference for ACPI when determining the server standards in the
mid-2010s, and so a few years since then, it could be helpful to
figure out whether those continue to make a difference, and how
Devicetree/FOSS driver implementations compare.
Post by Pete Batard
The Raspberry Pi SoC has also some very non-free code running that
executes prior to the running of the UEFI firmware and also in parallel
to the UEFI firmware and OS. There's basically a Management Engine
running on the GPU, which, among other things, provides a mailbox that
the UEFI firmware uses to retrieve or set hardware configuration data.
Yep, I've still got quite a lot to learn about that, I think.
Post by Pete Batard
On the other hand in this specific case, though I understand you're
speaking in a more general manner about ACPI usage, the whole ACPI blob
generation comes entirely from open source code.
I think that's great, and I've had a good experience with EDK2 +
Devicetree. My concerns about ACPI are basically that it seems like a
less-readable, larger-surface-area standard with more opaque processes
surrounding it. But I'm not an expert.
Post by Pete Batard
As far as I'm concerned, the main reason I wouldn't advocate ACPI +
Device Tree is that it can create user confusion as to what is really
being used behind the scenes. A bit like when PC end-users enable Legacy
boot in their UEFI settings and end up installing their OS in Legacy
mode on a UEFI capable system, then find themselves in a situation where
they want to use UEFI features but can't.
Getting slightly further off-topic, but for my own education: what is
an example of a UEFI feature that a user might want to use?

(to my mind, most of my preferences are: I want to use a standard,
easily-obtained installer, for each system install to be
straightforward and for the system to boot, for each system's devices
to function correctly, and for those to occur while minimizing the
amount of extraneous runtime code and I/O (roughly in that order of
precedence, and with FOSS practices helping a lot to achieve and
maintain those in the long-term). perhaps ranty, but it's intended to
explain why I don't love standards that include unusual and/or
unproven quirks, and binary blobs)
Post by Pete Batard
Also, again, since part of our goal has been to promote the emerging
SBBR standard (because we think it should ultimately help making
installation of various OSes on par with what is the case for x86 based
PCs, where you can pretty much just create a universal boot media as
well as have the OS install and use unified boot loaders, rather than
force users to concern themselves about the low-level system specifics
of their system, and play with yet another custom version of u-boot),
and SBBR made the choice to go with ACPI only rather than
ACPI+DeviceTree, we decided to propose ACPI+DeviceTree as a means not to
restrict user choice for people who want to use both, but knowing that
it's not something we really want to officially support.
Yep, that makes sense and your decision-making logic is sound.

If I were to imagine a two-line chart of device and hardware support
over time, with one line each representing the ACPI and Devicetree
approaches, then based on this standardization and industry backing, I
would expect ACPI to be the upper line for some duration of time
(decade?) until the point at which the drawbacks and disadvantages of
the extraneous functionality and complexity that I sense (but can't
really confirm, yet) outweigh the industry momentum placed behind it.

And I think that's a shame, because if the same effort and backing had
been placed behind a simpler, more open and maintainable approach
(namely the other line, Devicetree) then I think that overall
ecosystem cost savings and user/consumer respect improvements could
have been increased -- and that _should_ be good business. That's not
to say there aren't risks - if I were a strategic decision-maker with
investments/alignment in maintaining revenue models for some of the
affected technologies, processes and staff then I would probably feel
differently. Ultimately it feels like a question of time, though, and
delaying progress based on revenues and partnerships from a margin of
unnecessary cost passed on to purchasers feels wrong.
Post by Pete Batard
Post by James Addison
Slightly off-topic: do you know of cases where ACPI has helped a
vendor to adapt to shifting operating system interfaces or achieve
significant energy-usage savings?
I'm afraid I'm ill placed to discuss ACPI specifics, as, despite having
contributed one or two patches here and there to the Raspberry Pi UEFI
firmware ACPI bindings, my knowledge of ACPI is very limited.
With regards to adapting to OS interfaces my guess, even if it seems to
be at odds with the Linux kernel efforts of the past few years, is that
even with its limitations and constraints it was probably better to go
with a well established and relatively well supported implementation
that can effectively be shared with multiple OSes, rather than ask
hardware manufacturers to juggle with multiple implementations when
providing a functional implementation that complies with a specific
standard, especially a new one. So I would say that it was probably
decided that, since vendors were expected not to go with a standard that
would force them to "duplicate" some work, the burden of having to
support ACPI and only ACPI would be shifted to the OS folks, even if it
went against the direction that Linux had already been taking at that
stage...
My understanding is that the Linux development community has (and
continues to) pay the cost of maintaining ACPI compatibility, while
Devicetree (also with cost to support and develop) appears
well-supported across multiple architectures and appears (again, based
on very limited experience of mine) to be quite easy to work with,
configure and (re)deploy.
Post by Pete Batard
But again, this is pure speculation on my part, and I can't tell if
ultimately picking up ACPI only will actually help vendors to go with
SBBR and, hopefully, better adapt to shifting OS interfaces, since the
idea is that OSes should also strive to follow SBBR... even if that
means, in the case of Linux, having to support ACPI alongside Device
Tree. No idea about energy savings though.
Post by James Addison
I think that understanding some of
those could help to begin to address gaps that Debian/Linux/other
components have.
I suspect that if you post your question to the edk2-discuss mailing
list [1], which is frequented by the same developers as edk2-devel and
therefore should be seen by people who are very knowledgeable about
ACPI, you might be able to get insightful answers to your question.
I think there's a good chance that I've misunderstood a bunch of
important details already, so I'll take some more time to try to
figure out what those are and learn the landscape a little better --
and figure out whether there are problems that I can and want to help
to solve in that area -- before participating, but thank you.

Thanks,
James
Pete Batard
2023-05-08 17:40:01 UTC
Permalink
Well, I guess at this stage, and to help provide some more context about
the DT vs ACPI conundrum, I'm going to stop tiptoeing around the literal
elephant in the room, but not without first adding a preliminary notice
that I wasn't privy to whatever discussions occurred with regards to the
SBBR standard (I'm just a mere downstream developer, that got involved
well after SBBR was crafted), so part of it is yet again going to be
pure speculation.

The "elephant" in the room, in our case, is Microsoft since, from what I
understand, they did drive part of the design of the standard... which
of course makes sense since they do have an OS that they would like to
see people install on ARM servers and therefore will be interested in an
emerging ARM standard that's likely to affect them. And likewise, the
ARM folks, when devising a new standard, most certainly wanted feedback
from the company that still holds the lion's share of installed OSes out
there. So, any decision that was taken as part of the writing of the
standard is likely to have had Microsoft's footprint in one way or another.

That's not to say that (again from what I understand) Microsoft got free
reign, and that people interested in promoting the standard for Linux,
BSD or something else got the short end of the stick. But of course, now
that you know that Microsoft did play a part in this whole SBBR
endeavour, it might give you yet a different perspective on this whole
ACPI vs DT matter...
Post by James Addison
Somewhat agree, yep - although I think that in this case, there should
be various paths available (u-boot, EDK2, ACPI/DT), and if possible
I'd like to understand what is the approach that provides the most
compatibility and free software support with the fewest moving parts.
To me, the reliability and human-time cost savings from simpler, more
open and straightforward systems outweigh many other factors,
especially over the long term.
Agreed. But to others such as Microsoft, not having to integrate
something foreign when they already have good support for both UEFI and
ACPI in their OS was probably something that they had some strong
opinions about. In other words, while I like to imagine that Microsoft
did consider whether it would make sense to add support for DT for
Windows on ARM, and ultimately decided against it on a technical basis
rather than on an "it'll make life simpler for us", it appears that the
consensus settled on not having DT being part of the standard.
Post by James Addison
That's partly the reason I've been wondering about the power
consumption and operating-system-compatibility questions: as I
understand it, those were key reasons that server hardware vendors had
a preference for ACPI when determining the server standards in the
mid-2010s, and so a few years since then, it could be helpful to
figure out whether those continue to make a difference, and how
Devicetree/FOSS driver implementations compare.
If Microsoft and the people who drove the SBBR standard did review
things objectively, this might indeed be part of why they decided not to
add DT support.
Post by James Addison
I think that's great, and I've had a good experience with EDK2 +
Devicetree. My concerns about ACPI are basically that it seems like a
less-readable, larger-surface-area standard with more opaque processes
surrounding it. But I'm not an expert.
Yeah, neither am I, and I'll trust Linus and the kernel folks when they
decided that ACPI was such a liability that they had to devise something
better... This being said, I do wonder if DT has been holding all its
promises and doesn't come with its own pitfalls, now that we've seen it
used in practice for a few years. At least, as someone who has had to
help with implementing code that *alters* the original DT provided by
the Pi Foundation, so that it can actually be used downstream of UEFI
boot (which is something we have to do in the current Pi UEFI firmware
[1]) and has had to look from the sidelines at the annoyance of still
having to use a compiled-in format (couldn't some bz2 of plaintext, with
a little extra processing/runtime compilation from the kernel, work well
enough so that one could make editing of DT a lot friendlier for the end
users? Talk about doing away with binary formats...), I can't say I'm
entirely convinced that DT couldn't have been improved further.
Post by James Addison
Getting slightly further off-topic, but for my own education: what is
an example of a UEFI feature that a user might want to use?
Well, the big one would of course be Secure Boot.

Others would be the ability to have an interface from the OS, that
allows one to modify boot order/boot entries and read/write some
(relatively) standardised pre-boot environment variables. Oh and don't
forget the ESP, which is a lot more user friendly as a means to alter
your bootloaders and boot environment (just copy/edit at the file system
level, from your OS and with your utilities of choice, or use a full
blown shell if you need to) than using the (AFAIK) much more limited
means of accomplishing the same with something like u-boot.

Plus, UEFI has an official standard, and standards are (for the most
part) a good thing.
Post by James Addison
(to my mind, most of my preferences are: I want to use a standard,
easily-obtained installer,
I believe UEFI gives you that.
Post by James Addison
for each system install to be straightforward
Well, the standardised ESP and boot variables of UEFI should give you
that too (that is, as long as Microsoft doesn't abuse the ability to
change/alter the UEFI boot entries from the OS, in order to disregard
the user's choice and force Windows first, which they do tend to do).
Post by James Addison
and for the system to boot, for each system's devices
to function correctly,
Before OS handover, if your bootloader is UEFI compliant, it will have
access to all of the system's devices for which UEFI has a driver.
Post by James Addison
and for those to occur while minimizing the
amount of extraneous runtime code and I/O (roughly in that order of
precedence, and with FOSS practices helping a lot to achieve and
maintain those in the long-term).
I'd say UEFI drivers are usually fairly lean, at least for the base ones
provided by EDK2, even if supporting the UEFI standard may make it look
like they are encumbered with extraneous stuff. But that's standards for
you, and they exist to ensure that, if you do follow them, you're not
going to end up with something like a nasty Linux Shim boot loop for
instance (which is something that recently happened to me with a driver
I had put together, and that wasn't fully UEFI compliant [2])...
Post by James Addison
perhaps ranty, but it's intended to
explain why I don't love standards that include unusual and/or
unproven quirks, and binary blobs)
As we've seen above, I'd go further than that, and consider that DT
isn't exactly ideal in that respect, as it still, in essence, works with
binary blobs...

As to quirks, for having worked with the EDK2 folks, I'd say that if
there's something that they strive to avoid, it's unusual quirks or
anything that's left open to interpretation (which is another thing
that's caused me some grief with the RPi UEFI implementation, as we did
identify an element from the specs that was subject to interpretation,
and the specs were refined... but in a different direction than what we
would have wanted [3]).
Post by James Addison
If I were to imagine a two-line chart of device and hardware support
over time, with one line each representing the ACPI and Devicetree
approaches, then based on this standardization and industry backing, I
would expect ACPI to be the upper line for some duration of time
(decade?) until the point at which the drawbacks and disadvantages of
the extraneous functionality and complexity that I sense (but can't
really confirm, yet) outweigh the industry momentum placed behind it.
However, with what I have mentioned initially and the weight that
Microsoft has, the only way you're going to have vendors moving away
from ACPI is if Microsoft decides to add support for DT in Windows (or
something else that they see better than ACPI)... which I don't really
see happening in the near to medium term, since, much to their benefit,
Microsoft did manage to get the ARM SBBR specs to go with ACPI and thus
continue business as usual as far as Windows is concerned.
Post by James Addison
And I think that's a shame, because if the same effort and backing had
been placed behind a simpler, more open and maintainable approach
(namely the other line, Devicetree) then I think that overall
ecosystem cost savings and user/consumer respect improvements could
have been increased -- and that _should_ be good business.
Again, I am hoping that there was some consideration given by Microsoft
and other members of the SBBR committee as to whether it made sense to
go with Device Tree. One of the problems I would see however is that, I
doubt the Linux folks consulted with Microsoft when they introduced
Device Tree (then again why and how would they) to see if Microsoft had
some interest for using it in the long run, and if so, what they could
do to make it more palatable for Windows usage.

And that's the old problem of ecosystems being split on OS lines, when
it would be nice if we could look a bit further than that, with
Linuxfolks most likely not that eager to ask Microsoft if they have some
input on the direction they wanted to take when they introduced DT, and,
likewise, Microsoft unlikely to want to use some "Not Invented Here"
technologies like DT, even more so when you have the other elephant in
the room for the "Not Invented Here", i.e. Apple, which seems so adverse
to compromising with anybody that it provides the worst example to
follow such as, using their own version of EFI on x86 based hardware
rather than UEFI, and then dropping that altogether on their ARM based
silicon, to now use their own completely custom pre OS execution
environment. At this stage, I should probably add that it's going to be
fat chance to ever see Apple use SBBR on their hardware even, if on
paper, it should have been a perfect target for it and sure would have
make booting and installing Debian on Apple Silicon easier... even if
one were to be constrained to use ACPI over DT.
Post by James Addison
Ultimately it feels like a question of time, though, and
delaying progress based on revenues and partnerships from a margin of
unnecessary cost passed on to purchasers feels wrong.
To me, it looks like mostly a question of reducing development cost as
well as what one can get away with by not going out of their way to try
to talk and seek compromise with other parties. And, as usual, it's the
end-users that pay the price by having hardware and OSes that restrict
what they should be able to do with it, starting with their ability to
install the OS they prefer on modern ARM64 hardware.

And that's actually the reason why I think efforts like SBBR should be
supported, because they are trying to break the status quo, even if it
appears that Microsoft might have gotten what they wanted at the
detriment of Linux, and even if Apple are unlikely to ever want to touch
SBBR with a ten foot pole...
Post by James Addison
My understanding is that the Linux development community has (and
continues to) pay the cost of maintaining ACPI compatibility, while
Devicetree (also with cost to support and develop) appears
well-supported across multiple architectures
Well supported across architectures: yes.
Well supported across OSes: Unfortunately no, when you do consider Windows.

And that is the main issue here. SBBR is not just for Linux, or for
Windows. So, yes, someone, on the OS side, is likely to have to do extra
work, whose only purpose will be to allow people to install a completely
different OS. And yes, that sucks. But if it benefits end-users, it's
the price one has to pay.

Regards,

/Pete

[1]
https://github.com/tianocore/edk2-platforms/blob/master/Platform/RaspberryPi/Drivers/FdtDxe/FdtDxe.c#L470-L507
[2] https://github.com/rhboot/shim/issues/558#issuecomment-1490993221
[3] https://bugzilla.tianocore.org/show_bug.cgi?id=3336
Lennart Sorensen
2023-05-08 20:20:01 UTC
Permalink
Plus, UEFI has an official standard, and standards are (for the most part) a
good thing.
IEEE-1275 is a standard too.
However, with what I have mentioned initially and the weight that Microsoft
has, the only way you're going to have vendors moving away from ACPI is if
Microsoft decides to add support for DT in Windows (or something else that
they see better than ACPI)... which I don't really see happening in the near
to medium term, since, much to their benefit, Microsoft did manage to get
the ARM SBBR specs to go with ACPI and thus continue business as usual as
far as Windows is concerned.
Again, I am hoping that there was some consideration given by Microsoft and
other members of the SBBR committee as to whether it made sense to go with
Device Tree. One of the problems I would see however is that, I doubt the
Linux folks consulted with Microsoft when they introduced Device Tree (then
again why and how would they) to see if Microsoft had some interest for
using it in the long run, and if so, what they could do to make it more
palatable for Windows usage.
Well devicetree is part of open firmware aka IEEE-1275, from 1994.
ACPI is from 1996. And of course since I don't think Sun cared at
all what Microsoft thought when they designed the firmware interface,
I don't think that matters. Since open firmware or at least devicetree
has then ended up used on SPARC, PowerPC, ARM, MIPS, and who knows what
else, while ACPI is x86 and now some ARM but nothing else as far as I
can tell, other than in terms of overall units using it, ACPI certainly
has a lot less interest.
And that's the old problem of ecosystems being split on OS lines, when it
would be nice if we could look a bit further than that, with Linuxfolks most
likely not that eager to ask Microsoft if they have some input on the
direction they wanted to take when they introduced DT, and, likewise,
Microsoft unlikely to want to use some "Not Invented Here" technologies like
DT, even more so when you have the other elephant in the room for the "Not
Invented Here", i.e. Apple, which seems so adverse to compromising with
anybody that it provides the worst example to follow such as, using their
own version of EFI on x86 based hardware rather than UEFI, and then dropping
that altogether on their ARM based silicon, to now use their own completely
custom pre OS execution environment. At this stage, I should probably add
that it's going to be fat chance to ever see Apple use SBBR on their
hardware even, if on paper, it should have been a perfect target for it and
sure would have make booting and installing Debian on Apple Silicon
easier... even if one were to be constrained to use ACPI over DT.
Apple is almost certainly back to devicetree on their arm devices since
they used it on their powerpc based systems already in the past.
To me, it looks like mostly a question of reducing development cost as well
as what one can get away with by not going out of their way to try to talk
and seek compromise with other parties. And, as usual, it's the end-users
that pay the price by having hardware and OSes that restrict what they
should be able to do with it, starting with their ability to install the OS
they prefer on modern ARM64 hardware.
And that's actually the reason why I think efforts like SBBR should be
supported, because they are trying to break the status quo, even if it
appears that Microsoft might have gotten what they wanted at the detriment
of Linux, and even if Apple are unlikely to ever want to touch SBBR with a
ten foot pole...
Well unfortunately for years it seems the status quo has also been 64
bit arm servers being annonced, but not able to be purchased anywhere
unless you were google or amazon or some other large cloud provider.
Apparently they didn't actually want to sell their systems. Then they
complained that there was no market for arm servers. Maybe it has gotten
better in the last few years, but in my current job having arm servers
is no longer relevant unfortunately. I sure wanted some 8 or 10 years
ago when they were announcing them but not actually selling them.
Well supported across architectures: yes.
Well supported across OSes: Unfortunately no, when you do consider Windows.
Any OS that ever ran on an openfirmware system, which is a lot of them.
And that is the main issue here. SBBR is not just for Linux, or for Windows.
So, yes, someone, on the OS side, is likely to have to do extra work, whose
only purpose will be to allow people to install a completely different OS.
And yes, that sucks. But if it benefits end-users, it's the price one has to
pay.
It seems that it benefits only one OS, the one it seems just about no
one cares to run on arm systems.

Doesn't mean that long term ACPI and UEFI might not be better for SBBR.
I highly doubt the decision was made without heavy bias though.
--
Len Sorensen
Pete Batard
2023-05-08 23:40:02 UTC
Permalink
Post by Lennart Sorensen
Well devicetree is part of open firmware aka IEEE-1275, from 1994.
ACPI is from 1996.
Interesting; TIL.

I guess I'm probably not the only person who thought DT was something
that was only cooked recently by Linux kernel maintainers, since that's
when it became mainstream for the majority of the x86/PC based end-user
crowd.
Post by Lennart Sorensen
And of course since I don't think Sun cared at
all what Microsoft thought when they designed the firmware interface,
I don't think that matters. Since open firmware or at least devicetree
has then ended up used on SPARC, PowerPC, ARM, MIPS, and who knows what
else, while ACPI is x86 and now some ARM but nothing else as far as I
can tell, other than in terms of overall units using it, ACPI certainly
has a lot less interest.
Well, the thing is overall units tends to correlate pretty closely to
overall end-users. And, while one may try to plan for what one
expects/hopes the end-user landscape to shift towards, one must still
strive to create software that makes life easier for the current one. In
terms of units, ACPI has certainly been more widespread, even if it's
mostly due to the homogeneity of the dominant arch and platform (x86
based PC).
Post by Lennart Sorensen
Apple is almost certainly back to devicetree on their arm devices since
they used it on their powerpc based systems already in the past.
If that's the case, then (I'm going to assume that) it's shame Apple
didn't use their position to join the discussion and provide some
opposition to Microsoft, when it came to going with ACPI-only for SBBR...
Post by Lennart Sorensen
Well unfortunately for years it seems the status quo has also been 64
bit arm servers being annonced, but not able to be purchased anywhere
I'd venture to say that there's been a bit of a chicken and egg problem
there, which SBBR is in part trying to solve, by attempting to remove
one of the hurdle people face when getting an ARM64 server, i.e. how the
heck am I going to install my OS/distro of choice on this machine,
instead of being constrained by whatever custom version of some other
OS/distro the manufacturer of the platform decided to go with.

I do agree that we're still early in the game here, if game there
eventually is, which is the *precise* reason why a group of us worked
together to provide an SBBR implementation on the commonplace and
relatively limited Pi platforms, i.e. something that is everything but
an ARM server, to both provide an easy to access working implementation
as well as demonstrate to ARM64 system manufacturers that, if this can
be accomplished for the Pi with limited effort, this should certainly be
achievable for their platform.
Post by Lennart Sorensen
Post by Pete Batard
Well supported across architectures: yes.
Well supported across OSes: Unfortunately no, when you do consider Windows.
Any OS that ever ran on an openfirmware system, which is a lot of them.
Again, I'd prefer to go with number end-users as a better measure, since
we're not developing for the greater variety but for is likely to
benefit the greater masses. Of course, if ARM64 system manufacturers
collectively decide they want to ignore Windows and SBBR, and go with
openfirmware, I'll be more than happy to let Microsoft add openfirmware
compatibility to Windows on ARM, as long as the end-users stop having to
jump through system-specific hoops to install the OS they want.
Post by Lennart Sorensen
It seems that it benefits only one OS, the one it seems just about no
one cares to run on arm systems.
I'm going to go further than that and say that not even Microsoft
appears/appeared to care that much about running Windows on ARM systems,
as we pretty much offered them a golden opportunity to chase a goal they
should be exceedingly familiar with, and, what's more, one that Apple
will never challenge them on, namely running their OS on *cheap*
commonplace hardware. But they pretty much ignored the crowd that was
interested in running Windows on the Pi, even as Windows 11 21H2 does
run very decently on an 8 GB Pi 4 and, current hardware price hike
notwithstanding, could have gained significant traction with the low
income masses. This, in turn, could have provided Microsoft with a means
to bring (or should I say lock-in) those masses into the Windows
ecosystem. But instead, it appears they looked at what Apple was doing
with controlling both the hardware and software through an expensive
platform, and decided they wanted a piece of that pie, which I doubt
will benefit them much in terms of trying to keep Windows as the
dominant OS.

Now, as an aside, since the above may make it look like I'm trying to
advocate for Microsoft and we're on a GNU/Linux mailing list after all,
please bear in mind that, even as I am primarily a Windows developer
these days, I am far from rooting for Microsoft or any proprietary
software company for that matter. In fact, the main software I develop
on Windows is also designed to allow people break away from the
abomination that is closed-source proprietary software, like Windows,
*if they so choose* (since you cannot offer people freedom without also
giving them choice). As such, I am kind of supporting SBBR for Windows
on account that I expect both a server-targeted specification to
percolate to mainstream platforms eventually as well as the masses
running these end-user platforms to be initially more interested in
running Windows, if they have the choice, as opposed to some other OS.
From there, if we have something familiar like UEFI, I am hoping these
users may become more receptive to switching to a free OS. Oh, and for
the record, I have quite a different view when it comes to the interest
there has been with regards to running Windows on ARM, due to the direct
feedback we got once in turned out that the Pi UEFI firmware could be
used to get Windows to run on these platforms, even when it originally
did so with atrocious performance on the severely limited Pi 3. But of
course, that interest wasn't from people trying to run these platforms
as servers in the first place.
Post by Lennart Sorensen
Doesn't mean that long term ACPI and UEFI might not be better for SBBR.
I highly doubt the decision was made without heavy bias though.
Yeah, while I do hope that this wasn't the case, I too assume that there
weren't much of anybody in the room to try to counter Microsoft.

But again, I have no idea how ARM and SBBR actually came about, along
with who actually got invited to participate in the discussion, and who
might have declined to do so...

Regards,

/Pete
Lennart Sorensen
2023-05-09 14:00:01 UTC
Permalink
Post by Pete Batard
Interesting; TIL.
I guess I'm probably not the only person who thought DT was something that
was only cooked recently by Linux kernel maintainers, since that's when it
became mainstream for the majority of the x86/PC based end-user crowd.
You are definitely not the only one. The ability to attach an fdt
devicetree file to the kernel on systems that didn't have openfirmware
to provide the data was a more recent addition, but it was simply taking
advantage of a system that was already present for years.
Post by Pete Batard
Well, the thing is overall units tends to correlate pretty closely to
overall end-users. And, while one may try to plan for what one expects/hopes
the end-user landscape to shift towards, one must still strive to create
software that makes life easier for the current one. In terms of units, ACPI
has certainly been more widespread, even if it's mostly due to the
homogeneity of the dominant arch and platform (x86 based PC).
Certainly it is hard to fight against anything Microsoft and Intel have
decided they are going to use. Neither one seems to care much what
anyone else is doing already.
Post by Pete Batard
Post by Lennart Sorensen
Apple is almost certainly back to devicetree on their arm devices since
they used it on their powerpc based systems already in the past.
If that's the case, then (I'm going to assume that) it's shame Apple didn't
use their position to join the discussion and provide some opposition to
Microsoft, when it came to going with ACPI-only for SBBR...
Apple doesn't seem interested in their OS being able to run on anyone
else's hardware or anyone else's OS running on their hardware. So not
complyoing with standards probably suits them just fine.
Post by Pete Batard
I'd venture to say that there's been a bit of a chicken and egg problem
there, which SBBR is in part trying to solve, by attempting to remove one of
the hurdle people face when getting an ARM64 server, i.e. how the heck am I
going to install my OS/distro of choice on this machine, instead of being
constrained by whatever custom version of some other OS/distro the
manufacturer of the platform decided to go with.
Being able to install an OS certainly is handy, although being able to
even buy the hardware is probably the first step.
Post by Pete Batard
I do agree that we're still early in the game here, if game there eventually
is, which is the *precise* reason why a group of us worked together to
provide an SBBR implementation on the commonplace and relatively limited Pi
platforms, i.e. something that is everything but an ARM server, to both
provide an easy to access working implementation as well as demonstrate to
ARM64 system manufacturers that, if this can be accomplished for the Pi with
limited effort, this should certainly be achievable for their platform.
Certainly useful to have a small cheap platform to be able to work on it.
Unfortunate that even a Pi is a bit hard to get a hold of these days.
Post by Pete Batard
Again, I'd prefer to go with number end-users as a better measure, since
we're not developing for the greater variety but for is likely to benefit
the greater masses. Of course, if ARM64 system manufacturers collectively
decide they want to ignore Windows and SBBR, and go with openfirmware, I'll
be more than happy to let Microsoft add openfirmware compatibility to
Windows on ARM, as long as the end-users stop having to jump through
system-specific hoops to install the OS they want.
Well certainly devicetree makes the kernel and driver handling much
simpler and more consistent, but the boot loader on all the different
arm systems isn't standardized. Using UEFI on SBBR means a standard
grub2 uefi boot loader should work on any system, so that does seem
like a benefit. Of course that really just means UEFI might be a big
improvement, not that ACPI vs devicetree is. No idea if UEFI can work
with devicetree or not. Being an intel thing I would not be surprised
if ACPI is rather tied into it, since that would make sense. A quick
look at wikipedia says UEFI actually provides devicetree services on
RISC processor systems. I guess that makes sense since pretty much all
RISC systems seem to have used openfirmware or something similar so
their OS would expect that. Little endian only though of course.
Post by Pete Batard
I'm going to go further than that and say that not even Microsoft
appears/appeared to care that much about running Windows on ARM systems, as
we pretty much offered them a golden opportunity to chase a goal they should
be exceedingly familiar with, and, what's more, one that Apple will never
challenge them on, namely running their OS on *cheap* commonplace hardware.
But they pretty much ignored the crowd that was interested in running
Windows on the Pi, even as Windows 11 21H2 does run very decently on an 8 GB
Pi 4 and, current hardware price hike notwithstanding, could have gained
significant traction with the low income masses. This, in turn, could have
provided Microsoft with a means to bring (or should I say lock-in) those
masses into the Windows ecosystem. But instead, it appears they looked at
what Apple was doing with controlling both the hardware and software through
an expensive platform, and decided they wanted a piece of that pie, which I
doubt will benefit them much in terms of trying to keep Windows as the
dominant OS.
I am not sure why they are trying to make hardware. I get the impression
they make most of their money from subscription services these days,
and Azure.
Post by Pete Batard
Now, as an aside, since the above may make it look like I'm trying to
advocate for Microsoft and we're on a GNU/Linux mailing list after all,
please bear in mind that, even as I am primarily a Windows developer these
days, I am far from rooting for Microsoft or any proprietary software
company for that matter. In fact, the main software I develop on Windows is
also designed to allow people break away from the abomination that is
closed-source proprietary software, like Windows, *if they so choose* (since
you cannot offer people freedom without also giving them choice). As such, I
am kind of supporting SBBR for Windows on account that I expect both a
server-targeted specification to percolate to mainstream platforms
eventually as well as the masses running these end-user platforms to be
initially more interested in running Windows, if they have the choice, as
opposed to some other OS. From there, if we have something familiar like
UEFI, I am hoping these users may become more receptive to switching to a
free OS. Oh, and for the record, I have quite a different view when it comes
to the interest there has been with regards to running Windows on ARM, due
to the direct feedback we got once in turned out that the Pi UEFI firmware
could be used to get Windows to run on these platforms, even when it
originally did so with atrocious performance on the severely limited Pi 3.
But of course, that interest wasn't from people trying to run these
platforms as servers in the first place.
Yeah, while I do hope that this wasn't the case, I too assume that there
weren't much of anybody in the room to try to counter Microsoft.
I would expect Microsoft simply had no intension of adding anything new.
They already had UEFI and ACPI support, so why change it. Certainly it
is the interface of the most common platform out there, so using it does
make some sense.
Post by Pete Batard
But again, I have no idea how ARM and SBBR actually came about, along with
who actually got invited to participate in the discussion, and who might
have declined to do so...
Yeah I am sure someone knows.
--
Len Sorensen
Steve McIntyre
2023-05-09 14:30:01 UTC
Permalink
Post by Lennart Sorensen
Post by Pete Batard
Again, I'd prefer to go with number end-users as a better measure, since
we're not developing for the greater variety but for is likely to benefit
the greater masses. Of course, if ARM64 system manufacturers collectively
decide they want to ignore Windows and SBBR, and go with openfirmware, I'll
be more than happy to let Microsoft add openfirmware compatibility to
Windows on ARM, as long as the end-users stop having to jump through
system-specific hoops to install the OS they want.
Well certainly devicetree makes the kernel and driver handling much
simpler and more consistent, but the boot loader on all the different
arm systems isn't standardized. Using UEFI on SBBR means a standard
grub2 uefi boot loader should work on any system, so that does seem
like a benefit. Of course that really just means UEFI might be a big
improvement, not that ACPI vs devicetree is. No idea if UEFI can work
with devicetree or not. Being an intel thing I would not be surprised
if ACPI is rather tied into it, since that would make sense. A quick
look at wikipedia says UEFI actually provides devicetree services on
RISC processor systems. I guess that makes sense since pretty much all
RISC systems seem to have used openfirmware or something similar so
their OS would expect that. Little endian only though of course.
UEFI doesn't have to push ACPI, no. Indeed, some of the UEFI platforms
out there (e.g. Macchiatobin) let you choose whether to pass on DT or
ACPI to the boot loader / OS.

AIUI the main reason that Microsoft went with ACPI on the Arm platform
is just that they already had ACPI for x86. It's therefore much easier
for them to push the same requirements onto new platforms, of course.

DT for Arm is very much *not* just for Linux (see FreeBSD and other
OSes, and of course various boot loaders). However, there is still an
outstanding historical mess: lofs of Linux developers think that the
DT configuration on various platforms is theirs to change as they
please to suit the Linux kernel.

The original DT plan was to only ever include DT sources with the
Linux tree as a *temporary* thing until a reasonable number of
platforms provided DT data directly from firmware. DT was just meant
to be a static description *of the hardware*. But (like a lot of
"temporary" things), we're now a lot of years later and there's no
sign this is ever going to change. There's certainly no will to have a
fight about this.

Against that background, I genuinely think Microsoft have done the
sensible thing by sticking to ACPI rather than embracing DT for Arm
platforms...
--
Steve McIntyre, Cambridge, UK. ***@einval.com
Dance like no one's watching. Encrypt like everyone is.
- @torproject
Paul Wise
2023-05-09 05:50:01 UTC
Permalink
Post by James Addison
I'd like to understand what is the approach that provides the most
compatibility and free software support with the fewest moving parts.
To me, the reliability and human-time cost savings from simpler, more
open and straightforward systems outweigh many other factors,
especially over the long term.
As an end user, what I would like is everything above the CPU bootrom
should be open source, packaged within Debian main and auto-updated on
the appropriate schedule considering flash longevity and update types.

u-boot probably comes closest to that, although blobs are required on
some platforms, vendor forks are numerous and updates are manual only.

EDK2 is similar but worse because edk2-platforms is not available in
Debian and devices run eventually abandoned proprietary EDK2 forks.

ACPI comes from the world of proprietary non-redistribable vendor
firmware (some EDK2 forks) provided separately to the operating system.
Post by James Addison
Post by Pete Batard
The Raspberry Pi SoC has also some very non-free code running
Yep, I've still got quite a lot to learn about that, I think.
The LibreRPi folks are working on replacing that:

https://github.com/librerpi/lk-overlay/
--
bye,
pabs

https://wiki.debian.org/PaulWise
Loading...