Atomic 64-bit reads from mmap-ed memory on armel...

[Stéphane Glondu]

Hi all,

While investigating OCaml 5.2.0 build failure on armel, I realized that
atomic 64-bit reads from mmap-ed memory on armel results in a segfault.

See:

https://github.com/ocaml/ocaml/issues/13234#issuecomment-2176079951

for a minimal example.

Is this behaviour known? Is there a will to fix it?

A contributor suggests a problem in gcc (libatomic) and/or the kernel.


Cheers,

[Emanuele Rocca]

Hello Stéphane,

Post by Stéphane Glondu
While investigating OCaml 5.2.0 build failure on armel, I realized that
atomic 64-bit reads from mmap-ed memory on armel results in a segfault.
https://github.com/ocaml/ocaml/issues/13234#issuecomment-2176079951

In the reproducer above, try opening the file O_RDWR instead, and mmap
it with PROT_READ | PROT_WRITE. The program does not segfault if you do.

To see why, get the reproducer to segfault under gdb and inspect the
program counter, it will say something like:

(gdb) x/i $pc
=> 0xffff0f78: @ <UNDEFINED> instruction: 0x01a23e96

That instruction is stlexdeq, see [1].

So that's a *store* instruction, which may be a bit confusing in this
context given that in your program you are doing an atomic *read*
really. However, what the atomic read does is a pair of load/store
exclusive to guarantee atomicity of the 64 bit read. The page tables and
underlying storage both need to be writable for that to succeed.

The reason why you need a load/store exclusive pair to guarantee
atomicity of the 64 bit read is that the store exclusive ensures that
both 32 bit values have been read together. Otherwise the store
exclusive would fail and the routine would retry.
"Store-Release Exclusive Doubleword stores a doubleword from two
registers to memory if the executing PE has exclusive access to the
memory at that address, and returns a status value of 0 if the store was
successful, or of 1 if no store was performed."

[1] https://sources.debian.org/src/linux/6.7.12-1~bpo12%2B1/arch/arm64/kernel/kuser32.S/#L30
[2] https://developer.arm.com/documentation/ddi0597/2024-03/Base-Instructions/STLEXD--Store-Release-Exclusive-Doubleword-?lang=en

[Emanuele Rocca]

Hey,

Index: ocaml-5.2.0/otherlibs/runtime_events/runtime_events_consumer.c

Please ignore the diff, this was sent out by mistake as I was testing my
suggestions. I haven't checked any other area in the code, so I'm not
sure what a proper patch should look like, but hopefully you now have
enough information to come up with one. :-)

Emanuele

[Stéphane Glondu]

Post by Emanuele Rocca

Index: ocaml-5.2.0/otherlibs/runtime_events/runtime_events_consumer.c

Please ignore the diff, this was sent out by mistake as I was testing my
suggestions. I haven't checked any other area in the code, so I'm not
sure what a proper patch should look like, but hopefully you now have
enough information to come up with one. :-)

I was suspicious about the RO->RW move, so I've asked upstream:

https://github.com/ocaml/ocaml/issues/13234#issuecomment-2182751239

I've also launched a build (in my qemu-user-static chroot) with your
patch, just to see how it goes.

Thank you anyway! Your explanations are valuable.


Cheers,