Nopl: Difference between revisions

From JookWiki
(Display as lowercase nopl)
(Note about 64-bit systems)
Line 54: Line 54:
In 2006 [https://sourceware.org/pipermail/binutils/2006-June/047692.html PATCH: Add "nop memory" for i386/x86-64] was committed to the GNU Assembler. It added support for the 'nopl' and 'nopw' assembly instructions that map to multi-byte nop code.
In 2006 [https://sourceware.org/pipermail/binutils/2006-June/047692.html PATCH: Add "nop memory" for i386/x86-64] was committed to the GNU Assembler. It added support for the 'nopl' and 'nopw' assembly instructions that map to multi-byte nop code.


In 2007 [http://lkml.iu.edu/hypermail/linux/kernel/0709.2/2726.html x86: multi-byte single instruction NOPs] was committed to Linux. This added a set of 'P6 nops' that used the multi-byte nop opcodes and used them for i686 or newer x86 CPUs. Which type of nops to use were decided at runtime, so running an i686 kernel on an i586 machine would not cause any issues with this.  
In 2007 [http://lkml.iu.edu/hypermail/linux/kernel/0709.2/2726.html x86: multi-byte single instruction NOPs] was committed to Linux. This added a set of 'P6 nops' that used the multi-byte nop opcodes and used them for i686 or newer x86 CPUs. Which type of nops to use were decided at runtime, so running an i686 kernel on an i586 machine would not cause any issues with this. Strangely on 64-bit systems the nops were only used if your CPU vendor was Intel.  


In 2008 the Debian bugs [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=463606 Linux 2.6.24 fails to boot on MS Virtual PC 2007] and [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464962 -686 build uses long noops, that are unsupported by Transmeta Crusoe, immediate crash on boot] were reported, as well as the Linux. After a lot of discussion, the following CPUs were reported to not support multi-byte NOPs:
In 2008 the Debian bugs [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=463606 Linux 2.6.24 fails to boot on MS Virtual PC 2007] and [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464962 -686 build uses long noops, that are unsupported by Transmeta Crusoe, immediate crash on boot] were reported, as well as the Linux. After a lot of discussion, the following CPUs were reported to not support multi-byte NOPs:
Line 92: Line 92:
Unfortunately the developers found the GNU Assembler would add multi-byte NOPs all the time and Virtual PC would freeze during detection of multi-byte NOP support.
Unfortunately the developers found the GNU Assembler would add multi-byte NOPs all the time and Virtual PC would freeze during detection of multi-byte NOP support.


In the end they just threw their arms up and said 'no multi-byte NOPs at all on 32-bit x86'. As a result, the 'nopl' flag found in /proc/cpuinfo is always hidden on 32-bit x86 and always shown on 64-bit x86.
In the end they just threw their arms up and said 'no multi-byte NOPs at all on 32-bit x86. As a result, the 'nopl' flag found in /proc/cpuinfo is always hidden on 32-bit x86 and always shown on 64-bit x86.

Revision as of 09:37, 27 January 2022

During research for my Geode Repair projects, I found something incredibly weird so this page is my attempt to write it down.

Background

The AMD Geode LX800 CPU is an incredibly strange CPU. Despite being branded AMD, it's a descendant of the Cyrix MediaGX with a ton of features stapled on.

It implements the following instruction sets:

  • Pentium (i586)
  • Pentium Pro (i686)
  • MMX
  • Extended MMX
  • 3DNow!
  • Enhanced 3DNow!
  • Extra AMD Geode instructions

Notably the machine does not support PAE (Physical Address Extension). This may be why the CPUID identifies as family 5 (Pentium), not family 6 (Pentium Pro). This is all according to my reading of the AMD Geode LX Processors Data Book.

In contrast, some CPUs like the VIA C3 marked their CPU as family 6 without implementing the instruction set specified in the Pentium Pro Family Developer's Manual Volume 2. Most notably the 'CMOV' instruction.

But in theory, you could run an i686 system on the Geode without PAE. In practice it's a bit more complicated.

Undocumented i686 instructions

In 1995 Intel released the Pentium Pro and its developer manual documenting the entire instruction set.

By 1997 Christian Ludloff had created a map of 2 byte x86 opcodes. This confirmed Intel's documentation, but included a few unknown opcodes: 0F 34, 0F 35, as well as 0F 18 through 0F 1F.

In 1997 those first two opcodes 0F 34 and 0F 35 were determined to be Pentium II SYSENTER and SYSEXIT instructions despite Intel only documenting this later in 1999. It turns out the instructions were available on the Pentium Pro but broken. See SYSENTER, Where Are you? for a good summary of this situation.

Hilariously enough, the Geode LX800 identifies as a Pentium but supports the SYSENTER and SYSEXIT instructions introduced in the Pentium Pro and finally made useful in the Pentium II. Weird.

In 1998 Christian Ludloff documented in his updated map of 2 byte x86 opcodes that the 0F 18 through 0F 1F range of opcodes were hinting NOPs. The first being the 0F 18 opcode which maps to PREFETCHh instructions. I believe this information was documented first in the Intel Architecture Optimization Reference Manual.

Later in 2003 Christian Ludloff clarified in an email thread Undocumented opcodes (HINT_NOP) that these hinting NOPs were declared by Intel in their 1995 patent US5701442. The idea behind this patent from my reading is that you can encode a program written in another ISA as a series of opcodes that are run as NOPs on older machines and the new ISA on a newer machine.

I'm not sure why, but third party x86 CPUs aside from AMD didn't implement these NOPs. Perhaps Intel kept this patent close to their heart? Or maybe it's just not worth spending silicon and research on NOPs that nobody used?

The birth of multi-byte NOP

In 2006 Intel released an updated IA-32 Intel Architecture Software Developer's Manual. In addition to the standard one byte NOP op code, the multi-byte opcode was documented. This NOP could be from 2 to 9 bytes long, much longer than a single byte NOP. This is a pretty useful instruction for aligning code and data in memory. The only thing weird about this instruction is that it was marked as available on Pentium Pro and newer machines despite being documented in 2006. It turned out that Intel had recycled one of their hinting NOPs (0F 1F) as a new instruction.

Someone pointed this out on the Intel forum in the thread Multi-byte NOP opcode made official. They had tested and verified the feature and even pointed out that it works on AMD processors despite this instruction not being documented anywhere. They asked a few hard hitting questions to Intel:

  • Why was this opcode secret?
  • Why does it work on AMD CPUs?
  • Why does AMD recommend an opcode of 66 66 66 90 for multi-byte NOPs?

Intel got back to them with a 'this information is Intel Confidential and would require an NDA to discuss' reply. NOPs are definitely serious business.

I'm going to just go out and guess that AMD recommended the 66 90 opcode series because their CPUs optimized it and it worked on older machines. While with Intel their solution seems to be to recycle their trash.

In 2007 Symantec wrote a blog post x86 Fetch-Decode Anomalies showing that Intel's hinting NOP opcode they assigned to the multi-byte NOP actually will attempt to fetch memory (and even page fault) if you instruct it to. This isn't a problem in practice, but it gives more evidence this opcode isn't strictly a NOP.

Linux fallout

In 2006 PATCH: Add "nop memory" for i386/x86-64 was committed to the GNU Assembler. It added support for the 'nopl' and 'nopw' assembly instructions that map to multi-byte nop code.

In 2007 x86: multi-byte single instruction NOPs was committed to Linux. This added a set of 'P6 nops' that used the multi-byte nop opcodes and used them for i686 or newer x86 CPUs. Which type of nops to use were decided at runtime, so running an i686 kernel on an i586 machine would not cause any issues with this. Strangely on 64-bit systems the nops were only used if your CPU vendor was Intel.

In 2008 the Debian bugs Linux 2.6.24 fails to boot on MS Virtual PC 2007 and -686 build uses long noops, that are unsupported by Transmeta Crusoe, immediate crash on boot were reported, as well as the Linux. After a lot of discussion, the following CPUs were reported to not support multi-byte NOPs:

  • VIA C3 Nehemiah
  • Transmeta Crusoe TM5800
  • Microsoft Virtual PC 2007
  • QEMU 0.9.1
  • AMD Geode LX800

Interestingly enough the TM5800 reports its CPUID as family 5 like the LX800 does. But for the TM5800 the kernel promoted its status to i686 by changing the reported family at runtime.

The LX800 reports its family as 5, so shouldn't have failed due to the patch. I looked up other reports from the time and found the bug report 2.6.24-rc8 hangs at mfgpt-timer which seems to fit better, especially since the reporter didn't post any logs.

In parallel [BUG] x86 kenel won't boot under Virtual PC was reported to the Linux mailing list. This has a bit of a more focused discussion about how to address this problem. To summarize the discussion:

  • GNU Assembler shouldn't generate multi-byte NOPs if not all i686 CPUs support it
  • This is all too much effort for NOPs

A flurry of fixes appeared in Linux 2.6.25 to stop the bleeding:

This decided Transmeta Crusoe CPUs weren't i686, and limited use of multi-byte NOPs to i686 non-generic kernels.

A little while later in Linux 2.6.27 this amazing chain of patches happened:

Instead of only adding multi-byte NOPs to i686 and better machines during the kernel build, the plan was to only use multi-byte NOPs at runtime based on whether the CPU supported running multi-byte NOPs.

Unfortunately the developers found the GNU Assembler would add multi-byte NOPs all the time and Virtual PC would freeze during detection of multi-byte NOP support.

In the end they just threw their arms up and said 'no multi-byte NOPs at all on 32-bit x86. As a result, the 'nopl' flag found in /proc/cpuinfo is always hidden on 32-bit x86 and always shown on 64-bit x86.