Ability to further support 32bit architectures

[Dimitri John Ledkov]

Hi,

Hi
Linux 6.7 fails to build on at least i386 and armhf. Even it now
| cc1: out of memory allocating 135266296 bytes after a total of 235675648 bytes
Right now both fail on the same driver, so a short team workaround would
be to disable it. But we need a long term fix, and quickly.
As it is now, we will not be able to provide a kernel for maybe all
32bit architectures for Trixie.
Bastian

Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Separately, I wish we had cross-builders available, and cross-build
i386/armhf kernels from amd64/arm64 and thus having access to 64-bit
compiler.

I am experiencing the same issue with the armhf kernels on my infrastructure.

[Bastian Blank]

As it is now, we will not be able to provide a kernel for maybe all
32bit architectures for Trixie.

I don't think that this would be a reasonable decision. We're preparing to switch
32-bit architectures over to time64_t. Disabling 32-bit kernel builds would make
this whole work moot.

This is completely unrelated. Userland != kernel. And people already
talked about only supporting userland for those architectures.

FWIW, both m68k and powerpc are not affected by this bug, the powerpc build fails
because of a packaging problem.

Actually powerpc fails for exactly the same reason:

| cc1: out of memory allocating 135266296 bytes after a total of 244908032 bytes
| make[9]: *** [/<<PKGBUILDDIR>>/scripts/Makefile.build:248: drivers/media/pci/solo6x10/solo6x10-p2m.o] Error 1

https://buildd.debian.org/status/fetch.php?pkg=linux&arch=powerpc&ver=6.7-1%7Eexp1&stamp=1704796355&raw=0

Bastian

[Bastian Blank]

Hi

Post by Dimitri John Ledkov
Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Okay, maybe more workarounds exist. But none of them look really
promising.

Post by Dimitri John Ledkov
Separately, I wish we had cross-builders available, and cross-build
i386/armhf kernels from amd64/arm64 and thus having access to 64-bit
compiler.

Real cross-builders would use some fast amd64/arm64/ppc64el (and for
amd64 also reasonably cheap) machines to build all other architectures.

Bastian

[Adrian Bunk]

Post by Bastian Blank
...

Post by Dimitri John Ledkov
Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Okay, maybe more workarounds exist. But none of them look really
promising.
...

gcc being a memory hog on for C++ code is a hard problem,
and debug symbols for C++ code can be a problem since
they might be > 1 GB for some binaries.

But gcc needing more than 4 GB for a small C kernel driver is not
a problem for the "Ability to further support 32bit architectures",
that's a gcc bug that should be reported upstream just like you wouldn't
suggest dropping amd64 if gcc would ICE on one kernel driver on that
architecture.

Post by Bastian Blank
Bastian

cu
Adrian

[Aurelien Jarno]

Post by Adrian Bunk

Post by Bastian Blank
...

Post by Dimitri John Ledkov
Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Okay, maybe more workarounds exist. But none of them look really
promising.
...

gcc being a memory hog on for C++ code is a hard problem,
and debug symbols for C++ code can be a problem since
they might be > 1 GB for some binaries.
But gcc needing more than 4 GB for a small C kernel driver is not
a problem for the "Ability to further support 32bit architectures",
that's a gcc bug that should be reported upstream just like you wouldn't
suggest dropping amd64 if gcc would ICE on one kernel driver on that
architecture.

Or maybe just blame the kernel instead:
https://lore.kernel.org/lkml/CAHk-=whkGHOmpM_1kNgzX1UDAs10+UuALcpeEWN29EE0m-my=***@mail.gmail.com/

[Jeffrey Walton]

Post by Bastian Blank

Post by Dimitri John Ledkov
Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Okay, maybe more workarounds exist. But none of them look really
promising.

Also see <https://wiki.debian.org/ReduceBuildMemoryOverhead>.

Post by Bastian Blank

Post by Dimitri John Ledkov
Separately, I wish we had cross-builders available, and cross-build
i386/armhf kernels from amd64/arm64 and thus having access to 64-bit
compiler.

Real cross-builders would use some fast amd64/arm64/ppc64el (and for
amd64 also reasonably cheap) machines to build all other architectures.

Jeff

[YunQiang Su]

Keeping in mind that I am new to this arena...
I have some Intel systems - both 64-bit and 32-bit - that I might be able to use as build platforms.

I guess all of your hardwares are 64bit. You setup different OS on them.

What does the Debian team need from me to be able to use these systems?

It's not about performance of hardware. It is about some limitation of 32bit.
2 examples for it:
1. if we use 32bit value for time, it will overflow in 2038, then
your time will be shown as 1900.
https://en.wikipedia.org/wiki/Year_2038_problem
2. A single process (or maybe APP, not precisely), can only use UP
to 4GiB RAM.
In fact on most system the value is less than 4GiB:
on intel32, it is 3GiB
on mips32, it is 2GiB
But currently, it is not enough, for example, when we build a
big APP, it will need much more RAM.
The RAM does install in your Rack, but you can NOT use it.
https://en.wikipedia.org/wiki/3_GB_barrier

I can't guarantee they'll be FAST, but I'll do what I can to make them EFFECTIVE.

If you are really need 32bit system. Maybe you can say out why you
*REALLY* need it.
For most users, the suggestion is: upgrade to 64bit.

--J

[Alan Corey]

Are you forgetting that 64 bit is slower? In the arm world where it's
easily switchable 64 bit is pokey when you don't need it.

Sent from my mobile device.
------------------------------
*Sent:* Friday, January 12, 2024 10:11
*Subject:* Re: Ability to further support 32bit architectures

Keeping in mind that I am new to this arena...
I have some Intel systems - both 64-bit and 32-bit - that I might be

able to use as build platforms.
I guess all of your hardwares are 64bit. You setup different OS on them.
No. I have multiple 32-bit systems, one of which is Intel.

What does the Debian team need from me to be able to use these systems?

It's not about performance of hardware. It is about some limitation of 32bit.
1. if we use 32bit value for time, it will overflow in 2038, then
your time will be shown as 1900.
https://en.wikipedia.org/wiki/Year_2038_problem
2. A single process (or maybe APP, not precisely), can only use UP
to 4GiB RAM.
on intel32, it is 3GiB
on mips32, it is 2GiB
But currently, it is not enough, for example, when we build a
big APP, it will need much more RAM.
The RAM does install in your Rack, but you can NOT use it.
https://en.wikipedia.org/wiki/3_GB_barrier

I can't guarantee they'll be FAST, but I'll do what I can to make them

EFFECTIVE.
If you are really need 32bit system. Maybe you can say out why you
*REALLY* need it.
For most users, the suggestion is: upgrade to 64bit.
That's not at all what I was asking or talking about.
--J

[gene heskett]

Are you forgetting that 64 bit is slower?  In the arm world where it's
easily switchable 64 bit is pokey when you don't need it.

Thank Alan, to put numbers behind that, linuxcnc has a thing called
latency-test on an rpi4b the armhf kernel I built from 4.19 srcs, can
respond to an irq in 12 microseconds, this in 2018. in 2023, an arm64
rt-preempt kernel can only do that in around 50 microseconds. The huge
majority of that time is a increased size of the stack frame it has to
put away and then reload the environment to handle the interrupt.

Sent from my mobile device.
------------------------------------------------------------------------
*Sent:* Friday, January 12, 2024 10:11
*Subject:* Re: Ability to further support 32bit architectures
23:59写道：

Keeping in mind that I am new to this arena...
I have some Intel systems - both 64-bit and 32-bit - that I might

be able to use as build platforms.
I guess all of your hardwares are 64bit. You setup different OS on them.
No. I have multiple 32-bit systems, one of which is Intel.

What does the Debian team need from me to be able to use these

systems?
It's not about performance of hardware. It is about some limitation of 32bit.
   1. if we use 32bit value for time, it will overflow in 2038, then
your time will be shown as 1900.
https://en.wikipedia.org/wiki/Year_2038_problem
<https://en.wikipedia.org/wiki/Year_2038_problem>
   2. A single process (or maybe APP, not precisely), can only use UP
to 4GiB RAM.
          on intel32, it is 3GiB
          on mips32, it is 2GiB
       But currently, it is not enough, for example, when we build a
big APP, it will need much more RAM.
       The RAM does install in your Rack, but you can NOT use it.
https://en.wikipedia.org/wiki/3_GB_barrier
<https://en.wikipedia.org/wiki/3_GB_barrier>

I can't guarantee they'll be FAST, but I'll do what I can to make

them EFFECTIVE.
If you are really need 32bit system. Maybe you can say out why you
*REALLY* need it.
For most users, the suggestion is: upgrade to 64bit.
That's not at all what I was asking or talking about.
--J

Cheers, Gene Heskett.

[Alan Corey]

So, can you set an RPI4 to 32 bit for even more speed?
My Pi4 config.txt (Debian Bookworm) says

# Switch the CPU from ARMv7 into ARMv8 (aarch64) mode
arm_64bit=1

I haven't tried it because I don't know how the software will handle the
timing.

Post by gene heskett

Post by Alan Corey
Are you forgetting that 64 bit is slower? In the arm world where it's
easily switchable 64 bit is pokey when you don't need it.

Thank Alan, to put numbers behind that, linuxcnc has a thing called
latency-test on an rpi4b the armhf kernel I built from 4.19 srcs, can
respond to an irq in 12 microseconds, this in 2018. in 2023, an arm64
rt-preempt kernel can only do that in around 50 microseconds. The huge
majority of that time is a increased size of the stack frame it has to
put away and then reload the environment to handle the interrupt.

Post by Alan Corey
Sent from my mobile device.

------------------------------------------------------------------------

Post by Alan Corey
*Sent:* Friday, January 12, 2024 10:11
*Subject:* Re: Ability to further support 32bit architectures
23:59写道

Keeping in mind that I am new to this arena...
I have some Intel systems - both 64-bit and 32-bit - that I might

be able to use as build platforms.
I guess all of your hardwares are 64bit. You setup different OS on

them.

Post by Alan Corey
No. I have multiple 32-bit systems, one of which is Intel.

What does the Debian team need from me to be able to use these

systems?
It's not about performance of hardware. It is about some limitation of 32bit.
1. if we use 32bit value for time, it will overflow in 2038, then
your time will be shown as 1900.
https://en.wikipedia.org/wiki/Year_2038_problem
<https://en.wikipedia.org/wiki/Year_2038_problem>
2. A single process (or maybe APP, not precisely), can only use

UP

Post by Alan Corey
to 4GiB RAM.
on intel32, it is 3GiB
on mips32, it is 2GiB
But currently, it is not enough, for example, when we build a
big APP, it will need much more RAM.
The RAM does install in your Rack, but you can NOT use it.
https://en.wikipedia.org/wiki/3_GB_barrier
<https://en.wikipedia.org/wiki/3_GB_barrier>

I can't guarantee they'll be FAST, but I'll do what I can to make

them EFFECTIVE.
If you are really need 32bit system. Maybe you can say out why you
*REALLY* need it.
For most users, the suggestion is: upgrade to 64bit.
That's not at all what I was asking or talking about.
--J

Cheers, Gene Heskett.
--
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis

[Hank Barta]

Post by Alan Corey
So, can you set an RPI4 to 32 bit for even more speed?
My Pi4 config.txt (Debian Bookworm) says
# Switch the CPU from ARMv7 into ARMv8 (aarch64) mode
arm_64bit=1

You would have to try it to see if it is faster for your workload. I
suspect that the context switch is faster because there are fewer
registers to save/restore and they are 32 bit vs. 64 bit. Addresses
are also 32 bit, requiring less to load from RAM. (And I am not
familiar with ARM addressing modes so there may well be relative
addresses that allow shorter addresses in 64 bit mode.) I also wonder
about differences in CPU cache between 32 and 64 bit mode. Some of
these things slow specific operations while others (such as more
registers) can improve performance.

There's probably other stuff I'm not aware of that affects performance
comparisons between 64 and 32 bit mode.

[Bastian Blank]

Post by Dimitri John Ledkov
Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Disabling debug symbols does not help.

Bastian

Post by Dimitri John Ledkov
Sent from Ubuntu Pro
https://ubuntu.com/pro

Just curious why you send ads?

[Dimitri John Ledkov]

Heya,

Post by Bastian Blank

Post by Dimitri John Ledkov
Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Disabling debug symbols does not help.

This now smells a lot more like an actual bug in either kernel source code,
or compiler, or both. Rather than natural growth and actually needing that
much memory. Probably worth escalating.

Post by Bastian Blank
Bastian

Post by Dimitri John Ledkov
Sent from Ubuntu Pro
https://ubuntu.com/pro

Just curious why you send ads?

Felt cute, might remove later.

Post by Bastian Blank
--
Immortality consists largely of boredom.
-- Zefrem Cochrane, "Metamorphosis", stardate 3219.8

[Bastian Blank]

Post by Dimitri John Ledkov

Post by Bastian Blank

Post by Dimitri John Ledkov
Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Disabling debug symbols does not help.

This now smells a lot more like an actual bug in either kernel source code,
or compiler, or both. Rather than natural growth and actually needing that
much memory. Probably worth escalating.

What actually helps is -ftrack-macro-expansion=1.

https://salsa.debian.org/kernel-team/linux/-/merge_requests/998

Bastian