Muxup

Clarifying instruction semantics with P-Code

2024-02-20T12:00:00Z

I've recently had a need to step through quite a bit of disassembly for different architectures, and although some architectures have well-written ISA manuals it can be a bit jarring switching between very different assembly syntaxes (like "source, destination" for AT&T vs "destination, source" for just about everything else) or tedious looking up different ISA manuals to clarify the precise semantics. I've been using a very simple script to help convert an encoded instruction to a target-independent description of its semantics, and thought I may as well share it as well as some thoughts on its limitations.

instruction_to_pcode

The script is simplicity itself, thanks to the pypcode bindings to Ghidra's SLEIGH library which provides an interface to convert an input to the P-Code representation. Articles like this one provide an introduction and there's the reference manual in the Ghidra repo but it's probably easiest to just look at a few examples. P-Code is used as the basis of Ghidra's decompiler and provides a consistent human-readable description of the semantics of instructions for supported targets.

Here's an example aarch64 instruction:

$ ./instruction_to_pcode aarch64 b874c925
-- 0x0: ldr w5, [x9, w20, SXTW #0x0]
0) unique[0x5f80:8] = sext(w20)
1) unique[0x7200:8] = unique[0x5f80:8]
2) unique[0x7200:8] = unique[0x7200:8] << 0x0
3) unique[0x7580:8] = x9 + unique[0x7200:8]
4) unique[0x28b80:4] = *[ram]unique[0x7580:8]
5) x5 = zext(unique[0x28b80:4])

In the above you can see that the disassembly for the instruction is dumped, and then 5 P-Code instructions are printed showing the semantics. These P-Code instructions directly use the register names for architectural registers (as a reminder, AArch64 has 64-bit GPRs X0-X30 with the bottom halves acessible through W-W30). Intermediate state is stored in unique[addr:width] locations. So the above instruction sign-extends w20, adds to x9, and reads a 32-bit value from the resulting address, then zero-extends to 64-bits when storing into x5.

The output is somewhat more verbose for architectures with flag registers, e.g. cmpb $0x2f,-0x1(%r11) produces:

./instruction_to_pcode x86-64 --no-reverse-input "41 80 7b ff 2f"
-- 0x0: CMP byte ptr [R11 + -0x1],0x2f
0) unique[0x3100:8] = R11 + 0xffffffffffffffff
1) unique[0xbd80:1] = *[ram]unique[0x3100:8]
2) CF = unique[0xbd80:1] < 0x2f
3) unique[0xbd80:1] = *[ram]unique[0x3100:8]
4) OF = sborrow(unique[0xbd80:1], 0x2f)
5) unique[0xbd80:1] = *[ram]unique[0x3100:8]
6) unique[0x28e00:1] = unique[0xbd80:1] - 0x2f
7) SF = unique[0x28e00:1] s< 0x0
8) ZF = unique[0x28e00:1] == 0x0
9) unique[0x13180:1] = unique[0x28e00:1] & 0xff
10) unique[0x13200:1] = popcount(unique[0x13180:1])
11) unique[0x13280:1] = unique[0x13200:1] & 0x1
12) PF = unique[0x13280:1] == 0x0

But simple instructions that don't set flags do produce concise P-Code:

$ ./instruction_to_pcode riscv64 "9d2d"
-- 0x0: c.addw a0,a1
0) unique[0x15880:4] = a0 + a1
1) a0 = sext(unique[0x15880:4])

Other approaches

P-Code was an intermediate language I'd encountered before and of course benefits from having an easy to use Python wrapper and fairly good support for a range of ISAs in Ghidra. But there are lots of other options - angr (which uses Vex, taken from Valgrind) compares some options and there's more in this paper. Radare2 has ESIL, but while I'm sure you'd get used to it, it doesn't pass the readability test for me. The rev.ng project uses QEMU's TCG. This is an attractive approach because you benefit from more testing and ISA extension support for some targets vs P-Code (Ghidra support is lacking for RVV, bitmanip, and crypto extensions).

Another route would be to pull out the semantic definitions from a formal spec (like Sail) or even an easy to read simulator (e.g. Spike for RISC-V). But in both cases, definitions are written to minimise repetition to some degree, while when expanding the semantics we prefer explicitness, so would want to expand to a form that differs a bit from the Sail/Spike code as written.

Reflections on ten years of LLVM Weekly

2024-01-01T12:00:00Z

Today, with Issue #522 I'm marking ten years of authoring LLVM Weekly, a newsletter summarising developments on projects under the LLVM umbrella (LLVM, Clang, MLIR, Flang, libcxx, compiler-rt, lld, LLDB, ...). Somehow I've managed to keep up an unbroken streak, publishing every single Monday since the first issue back on Jan 6th 2014 (the first Monday of 2014 - you can also see the format hasn't changed much!). With a milestone like that, now is the perfect moment to jot down some reflections on the newsletter and thoughts for the future.

Motivation and purpose

Way back when I started LLVM Weekly, I'd been working with LLVM for a few years as part of developing and supporting a downstream compiler for a novel research architecture. This was a very educational yet somewhat lonely experience, and I sought to more closely follow upstream LLVM development to keep better abreast of changes that might impact or help my work, to learn more about parts of the compiler I wasn't actively using, and also to feel more of a connection to the wider LLVM community given my compiler work was a solo effort. The calculus for kicking off an LLVM development newsletter was dead simple: I found value in tracking development anyway, the incremental effort to write up and share with others wasn't too great, and I felt quite sure others would benefit as well.

Looking back at my notes (I have a huge Markdown file with daily notes going back to 2011 - a file of this rough size and format is also a good stress test for text editors!) it seems I thought seriously about the idea of starting something up at the beginning of December 2013. I brainstormed the format, looked at other newsletters I might want to emulate, and went ahead and just did it starting in the new year. It really was as simple as that. I figured better to give it a try and stop it if it gets no traction rather than waste lots of time putting out feelers on level of interest and format. As a sidenote, I was delighted to see many of the newsletters I studied at the time are still going: This Week in Rust Perl Weekly (I'll admit this surprised me!), Ubuntu Weekly News, OCaml Weekly News, and Haskell Weekly News.

Readership and content

The basic format of LLVM Weekly is incredibly simple - highlight relevant news articles and blog posts, pick out some forum/mailing discussions (sometimes trying to summarise complex debates - but this is very challenging and time intensive), and highlight some noteworthy commits from across the project. More recently I've taken to advertising the scheduled online sync-ups and office hours for the week. Notably absent are any kind of ads or paid content. I respect that others have made successful businesses in this kind of space, but although I've always written LLVM Weekly on my own personal time I've never felt comfortable trying to monetise other people's attention or my relationship with the community in this way.

The target audience is really anyone with an interest in keeping track of LLVM development, though I don't tend to expand every acronym or give a from-basics explanation for every term, so some familiarity with the project is assumed if you want to understand every line. The newsletter is posted to LLVM's Discourse, to llvmweekly.org, and delivered direct to people's inboxes. I additionally post on Twitter and on Mastodon linking to each issue. I don't attempt to track open rates or have functioning analytics, so only have a rough idea of readership. There are ~3.5k active subscribers directly to the mailing list, ~7.5k Twitter followers, ~180 followers on Mastodon (introduced much more recently), and an unknown number of people reading via llvmweekly.org or RSS. I'm pretty confident that I'm not just shouting in the void at least.

There are some gaps or blind spots of course. I make no attempt to try to link to patches that are under-review, even though many might have interesting review discussions because it would simply be too much work to sort through them and if the discussion is particularly contentious or requires input from a wider cross-section of the LLVM community you'd expect an RFC to be posted anyway. Although I do try to highlight MLIR threads or commits, as it's not an area of LLVM I'm working right now I probably miss some things. Thankfully Javed Absar has taken up writing an MLIR newsletter that helps plug those gaps. I'm also not currently trawling through repos under the LLVM GitHub organisation other than the main llvm-project monorepo, though perhaps I should...

I've shied away from reposting job posts as the overhead is just too high. I found dealing with requests to re-advertise (and considering if this is useful to the community) or determining if ads are sufficiently LLVM related just wasnt a good use of time when there's a good alternative. People can check the job post category on LLVM discourse or search for LLVM on their favourite jobs site.

How it works

There are really two questions to be answered here: how I go about writing it each week, and what tools and services are used. In terms of writing:

I have a checklist I follow just to ensure nothing gets missed and help dive back in quickly if splitting work across multiple days.
tig --since=$LAST_WEEK_DATE $DIR to step through commits in the past week for each sub-project within the monorepo. Tig is a fantastic text interface for git, and I of course have an ugly script that I bind to a key that generates the [shorthash](github_link) style links I insert for each chosen commit.
I make a judgement call as to whether I think a commit might be of interest to others. This is bound to be somewhat flawed, but hopefully better than ramdom selection! I always really appreciate feedback if you think I missed something important, or tips on things you think I should include next week.
- There's a cheat that practically guarantees a mention in LLVM Weekly without even needing to drop me a note though - write documentation! It's very rare I see a commit that adds more docs and fail to highlight it.
Similarly, I scan through LLVM Discourse posts over the past week and pick out discussions I think readers may be interested in. Most RFCs will be picked up as part of this. In some cases if there's a lengthy discussion I might attempt to summarise or point to key messages, but honestly this is rarer than I'd like as it can be incredibly time consuming. I try very hard to remain a neutral voice and no to insert personal views on technical discussions.
Many ask how long it takes to write, and the answer is of course that it varies. It's easy to spend a lot of time trying to figure out the importance of commits or discussions in parts of the compiler I don't work with much, or to better summarise content. The amount of activity can also vary a lot week to week (especially on Discourse). It's mostly in the 2.5-3.5h range (very rarely any more than 4 hours) to write, copyedit, and send.

There's not much to be said on the tooling said, except that I could probably benefit from refreshing my helper scripts. Mail sending is handled by Mailgun, who have changed ownership three times since I started. I handle double opt-in via a simple Python script on the server-side and mail sending costs me $3-5 a month. Otherwise, I generate the static HTML with some scripts that could do with a bit more love. The only other running costs are the domain name fees and a VPS that hosts some other things as well, so quite insignificant compared to the time commitment.

How you can help

I cannot emphasise enough that I'm not an expert on all parts of LLVM, and I'm also only human and can easily miss things. If you did something you think people may be interested in and I failed to cover it, I almost certainly didn't explicitly review it and deem it not worthy. Please do continue to help me out by dropping links and suggestions. Writing commit messages that make it clear if a change has wider impact also helps increase the chance I'll pick it up.

I noted above that it is particularly time consuming to summarise back and forth in lengthy RFC threads. Sometimes people step up and do this and I always try to link to it when this happens. The person who initiated a thread or proposal is best placed write such a summary, and it's also a useful tool to check that you interpreted people's suggestions/feedback correctly, but it can still be helpful if others provide a similar service.

Many people have fed back they find LLVM Weekly useful to stay on top of LLVM developments. This is gratifying, but also a pretty huge responsibility. If you have thoughts on things I could be doing differently to serve the community even better without a big difference in time commitment, I'm always keen to hear ideas and suggestions.

Miscellaneous thoughts

To state the obvious, ten years is kind of a long time. A lot has happened with me in that time - I've got married, we had a son, I co-founded and helped grow a company, and then moved on from that, kicked off the upstream RISC-V LLVM backend, and much more. One of the things I love working with compilers is that there's always new things to learn, and writing LLVM Weekly helps me learn at least a little more each week in areas outside of where I'm currently working. There's been a lot of changes in LLVM as well. Off the top off my head: there's been the move from SVN to Git, moving the Git repo to GitHub, moving from Phabricator to GitHub PRs, Bugzilla to GitHub issues, mailing lists to Discourse, relicensing to Apache 2.0 with LLVM exception, the wider adoption of office hours and area-specific sync-up calls, and more. I think even the LLVM Foundation was set up a little bit after LLVM Weekly started. It's comforting to see the llvm.org website design remains unchanged though!

It's also been a time period where I've become increasingly involved in LLVM. Upstream work - most notably initiating the RISC-V LLVM backend, organising an LLVM conference, many talks, serving on various program committees for LLVM conferences, etc etc. When I started I felt a lot like someone outside the community looking in and documenting what I saw. That was probably accurate too, given the majority of my work was downstream. While I don't feel like an LLVM "insider" (if such a thing exists?!), I certainly feel a lot more part of the community than I did way back then.

An obvious question is whether there are other ways of pulling together the newsletter that are worth pursuing. My experience with large language models so far has been that they haven't been very helpful in reducing the effort for the more time consuming aspects of producing LLVM Weekly, but perhaps that will change in the future. If I could be automated away then that's great - perhaps I'm misjudging how much of my editorial input is signal rather than just noise, but I don't think we're there yet for AI. More collaborative approaches to producing content would be another avenue to explore. For the current format, the risk is that the communication overhead and stress of seeing if various contributions actually materialise before the intended publication date is quite high. If I did want to spread the load or hand it over, then a rotating editorship would probably be most interesting to me. Even if multiple people contribute, each week a single would act as a backstop to make sure something goes out.

The unbroken streak of LLVM Weekly editions each Monday has become a bit totemic. It's certainly not always convenient having this fixed commitment, but it can also be nice to have this rhythm to the week. Even if it's a bad week, at least it's something in the bag that people seem to appreciate. Falling into bad habits and frequently missing weeks would be good for nobody, but I suspect that a schedule that allowed the odd break now and then would be just fine. Either way, I feel a sense of relief having hit the 10 year unbroken streak. I don't intend to start skipping weeks, but should life get in the way and the streak gets broken I'll feel rather more relaxed about it having hit that arbitrary milestone.

Looking forwards and thanks

So what's next? LLVM Weekly continues, much as before. I don't know of I'll still be writing it in another 10 years time, but I'm not planning to stop soon. If it ceases to be a good use of my time, ceases to have values for others, or I find there's a better way of generating similar impact then it would only be logical to move on. But for now, onwards and upwards.

Many thanks are due. Thank you to the people who make LLVM what it is - both technically and in terms of its community that I've learned so much from. Thank you to Igalia where I work for creating an environment where I'm privileged enough to be paid to contribute upstream to LLVM (get in touch if you have LLVM needs!). Thanks to my family for ongoing support and of course putting up with the times my LLVM Weekly commitment is inconvenient. Thank you to everyone who has been reading LLVM Weekly and especially those sending in feedback or tips or suggestions for future issues.

On a final note, if you've got this far you should make sure you are subscribed to LLVM Weekly and follow on Mastodon or on Twitter.

Let the (terminal) bells ring out

2023-12-24T12:00:00Z

I just wanted to take a few minutes to argue that the venerable terminal bell is a helpful and perhaps overlooked tool for anyone who does a lot of their work out of a terminal window. First, an important clarification. Bells ringing, chiming, or (as is appropriate for the season) jingling all sounds very noisy - but although you can configure your terminal emulator to emit a sound for the terminal bell, I'm actually advocating for configuring a non-intrusive but persistent visual notification.

BEL

Our goal is to generate a visual indicator on demand (e.g. when a long-running task has finished) and to do so with minimal fuss. This should work over ssh and without worrying about forwarding connections to some notification daemon. The ASCII BEL control character (alternatively written as BELL by those willing to spend characters extravagantly) meets these requirements. You'll just need co-operation from your terminal emulator and window manager to convert the bell to an appropriate notification.

BEL is 7 in ASCII, but can be printed using \a in printf (including the /usr/bin/printf you likely use from your shell, defined in POSIX). There's even a Rosetta Code page on ringing the terminal bell from various languages. Personally, I like to define a shell alias such as:

alias bell="printf '\aBELL!\n'"

Printing some text alongside the bell is helpful for confirming the bell was triggered as expected even after it was dismissed. Then, if kicking off a long operation like an LLVM compile and test use something like:

cmake --build . && ./bin/llvm-lit -s test; bell

The ; ensures the bell is produced regardless of the exit code of the previous commands. All being well, this sets the urgent hint on the X11 window used by your terminal, and your window manager produces a subtle but persistent visual indicator that is dismissed after you next give focus to the source of the bell. Here's how it looks for me in DWM:

The above example shows 9 workspaces (some of them named), where the llvm workspace has been highlighted because a bell was produced there. You'll also spot that I have a timers workspace, which I tend to use for miscellaneous timers. e.g. a reminder before a meeting is due to start, or when I'm planning to switch a task. I have a small tool for this I might share in a future post.

A limitation versus triggering freedesktop.org Desktop Notifications is that there's no payload / associated message. For me this isn't a big deal, such messages are distracting, and it's easy enough to see the full context when switching workspaces. It's possible it's a problem for your preferred workflow of course.

You could put \a in your terminal prompt ($PS1), meaning a bell is triggered after every command finishes. For me this would lead to too many notifications for commands I didn't want to carefully monitor the output for, but your mileage may vary. After publishing this article, my Igalia colleague Adrian Perez pointed me to a slight variant on this that he uses: in Zsh $TTYIDLE makes it easy to configure behaviour based on the duration of a command and he configures zsh so a bell is produced for commands that take longer than 30 seconds to complete.

Terminal emulator support

Unfortunately, setting the urgent hint upon a bell is not supported by gnome-terminal, with a 15 year-old issue left unresolved. It is however supported by the otherwise very similar xfce4-terminal (just enable the visual bell in preferences), and I switched solely due to this issue.

From what I can tell, this is the status of visual bell support via setting the X11 urgent hint:

xfce4-terminal: Supported. In Preferences -> Advanced ensure "Visual bell" is ticked.
xterm: Set XTerm.vt100.bellIsUrgent: true in your .Xresources file.
rxvt-unicode (urxvt): Set URxvt.urgentOnBell: true in your .Xresources file.
alacritty: Supported. Works out of the box with no additional configuration needed.
gnome-terminal: Not supported.
konsole: As far as I can tell it isn't supported. Creating a new profile and setting the "Terminal bell mode" to "Visual Bell" doesn't seem to result in the urgent hint being set.

Article changelog

2023-12-24: Add note about configuring a bell for commands taking longer than a certain threshold duration in Zsh.
2023-12-24: Initial publication date.

Ownership you can count on

2023-12-20T12:00:00Z

Introduction

I came across the paper Ownership You Can Count On (by Adam Dingle and David F. Bacon, seemingly written in 2007) some years ago and it stuck with me as being an interesting variant on traditional reference counting. Since then I've come across references to it multiple times in the programming language design and implementation community and thought it might be worth jotting down some notes on the paper itself and the various attempts to adopt its ideas.

Ownership you can count on and the Gel language

The basic idea is very straight-forward. Introduce the idea of an owning pointer type (not dissimilar to std::unique_ptr in C++) where each object may have only a single owning pointer and the object is freed when that pointer goes out of scope. In addition to that allow an arbitrary number of non-owning pointers to the object, but require that all non-owning pointers have been destroyed by the time the object is freed. This requirement prevents use-after-free and is implemented using run-time checks. A reference count is incremented or decremented whenever a non-owning pointer is created or destroyed (referred to as alias counting in the paper). That count is checked when the object is freed and if it still has non-owning references (i.e. the count is non-zero), then the program exits with an error.

Contrast this with a conventional reference counting system, where objects are freed when their refcount reaches zero. In the alias counting scheme, the refcount stored on the object is simply decremented when a non-owning reference goes out of scope, and this refcount needs to be compared to 0 only upon the owning pointer going out of scope rather than upon every decrement (as an object is never freed as a result of the non-owning pointer count reaching zero). Additionally, refcount manipulation is never needed when passing around the owning pointer. The paper also describes an analysis that allows many refcount operations to be elided in regions where there aren't modifications to owned pointers of the owned object's subclass/class/superclass (which guarantees the pointed-to object can't be destructed in this region). The paper also claims support for destroying data structures with pointer cycles that can't be automatically destroyed with traditional reference counting. The authors suggest for cases where you might otherwise reach for multiple ownership, (e.g. an arbitrary graph) to allocate an array of owning pointers to hold your nodes, then use non-owning pointers between them.

The paper describes a C# derived language called Gel which only requires two additional syntactic constructs to support the alias counting model: owned pointers indicated by a ^ (e.g. Foo ^f = new Foo(); and a take operator to take ownership of a value from an owning field. Non-owning pointers are written just as Foo f. They also achieve a rather neat erasure property, whereby if you take a Gel program and remove all ^ and take you'll have a C# program that is valid as long as the original Gel program was valid.

That all sounds pretty great, right? Does this mean we can have it all: full memory safety, deterministic destruction, low runtime and annotation overhead? As you'd expect, there are some challenges:

The usability in practice is going to depend a lot on how easy it is for a programmer to maintain the invariant that no unowned pointers outlive the owned pointer, especially as failure to do so results in a runtime crash.
- Several languages have looked at integrating the idea, so there's some information later in this article on their experiences.
Although data structures with pointer cycles that couldn't be automatically destroyed by reference counting an be handled, there are also significant limitations. Gel can't destroy graphs containing non-owning pointers to non-ancestor nodes unless the programmer writes logic to null out those pointers. This could be frustrating and error-prone to handle.
- The authors present a multi-phase object destruction mechanism aiming to address these limitations, though the cost (potentially recursively descending the graph of the ownership tree 3 times) depends on how much can be optimised away.
Although it's not a fundamental limitation of the approach, Gel doesn't yet provide any kind of polymorphism between owned and unowned references. This would be necessary for any modern language with support for generics.
The reference count elimination optimisation described in the paper assumes single-threaded execution.
- Though as noted there, thread escape analysis or structuring groups of objects into independent regions (also see recent work on Verona) could provide a solution.

So in summary, an interesting idea that is meaningfully different to traditional reference counting, but the largest program written using this scheme is the Gel compiler itself and many of the obvious questions require larger scale usage to judge the practicality of the scheme.

Influence and adoption of the paper's ideas

Ownership You Can Count On was written around 2007 and as far as I can tell never published in the proceedings of a conference or workshop, or in a journal. Flicking through the Gel language repository and applying some world-class logical deduction based on the directory name holding a draft version of the paper leads to me suspect it was submitted to PLDI though. Surprisingly it has no academic citations, despite being shared publicly on David F. Bacon's site (and Bacon has a range of widely cited papers related to reference counting / garbage collection). Yet, the work has been used as the basis for memory management in one language (Inko) and was seriously evaluated (partially implemented?) for both Nim and Vale.

Inko started out with a garbage collector, but its creator Yorick Peterse announced in 2021 a plan to adopt the scheme from Ownership You Can Count on, who then left his job to work on Inko full-time and successfully transitioned to the new memory management scheme in the Inko 0.10.0 release about a year later. Inko as a language is more ambitious than Gel - featuring parametric polymorphism, lightweight processes for concurrency, and more. Yet it's still early days and it's not yet, for instance, good for drawing conclusions about performance on modern systems as optimisations aren't currently applied. Dusty Phillips wrote a blog post earlier this year explaining Inko's memory management through some example data structures, which also includes some thoughts on the usability of the system and some drawbacks. Some of the issues may be more of a result of the language being young, e.g. the author notes it took a lot of trial and error to figure out some of the described techniques (perhaps this will be easier once common patterns are better documented and potentially supported by library functions or syntax?), or that debuggability is poor when the program exits with a dangling reference error.

Nim was at one point going to move to Gel's scheme (see the blog post and (RFC). I haven't been able to find a detailed explanation for the reasons why it was rejected, though Andreas Rumpf (Nim language creator) commented on a Dlang forum discussion thread about the paper that "Nim tried to use this in production before moving to ORC and I'm not looking back, 'ownership you can count on' was actually quite a pain to work with...". Nim has since adopted a more conventional non-atomic reference counted scheme (ARC), with an optional cycle collector (ORC).

Vale was previously adopting the Gel / Ownership You Can Count On scheme (calling the unowned references "constraint references"), but has since changed path slightly and now uses "generational references" Rather than a reference count, each allocation includes a generation counter which is incremented each time it is reused. Fat pointers that include the expected value of the generation counter are used, and checked before dereferencing. If they don't match, that indicates the memory location was since reallocated and the program will fault. This also puts restrictions on the allocator's ability to reuse freed allocations without compromising safety. Vale's memory management scheme retains similarities to Gel: the language is still based around single ownership and this is exploited to elide checks on owning references/pointers.

Some tangentially related things that didn't make it into the main body of text above:

A language that references Gel in its design but goes in a slightly different direction is Wouter van Oortmerssen's Lobster. Its memory management scheme attempts to infer ownership (and optimise in the case where single ownership can be inferred) rather than requiring single ownership like the languages listed above.
One of the discussions on Ownership You Can Count On referenced this Pointer Ownership Model paper as having similarities. It chooses to categorise pointers as either "responsible" or "irresponsible" - cute!

And, that's about it. If you're hoping for a definitive answer on whether alias counting is a winning idea or not I'm sorry to disappoint, but I've at least succeeded in collecting together the various places I've seen it explored and am looking forward to seeing how Inko's adoption of it evolves. I'd be very interested to hear any experience reports of adopting alias counting or using a language like Inko that tries to use it.

Storing data in pointers

2023-11-26T12:00:00Z

Introduction

On mainstream 64-bit systems, the maximum bit-width of a virtual address is somewhat lower than 64 bits (commonly 48 bits). This gives an opportunity to repurpose those unused bits for data storage, if you're willing to mask them out before using your pointer (or have a hardware feature that does that for you - more on this later). I wondered what happens to userspace programs relying on such tricks as processors gain support for wider virtual addresses, hence this little blog post. TL;DR is that there's no real change unless certain hint values to enable use of wider addresses are passed to mmap, but read on for more details as well as other notes about the general topic of storing data in pointers.

Storage in upper bits assuming 48-bit virtual addresses

Assuming your platform has 48-bit wide virtual addresses, this is pretty straightforward. You can stash whatever you want in those 16 bits, but you'll need to ensure you masking them out for every load and store (which is cheap, but has at least some cost) and would want to be confident that there's no other attempted users for these bits. The masking would be slightly different in kernel space due to rules on how the upper bits are set:

x86-64 defines a canonical form of addresses (see 3.3.7.1). This describes how on an implementation with 48-bit virtual addresses, bits 63:48 must be set to the value of bit 47 (i.e. sign extended). The memory map used by the Linux kernel uses bit 47 to split the address space between kernel and user addresses, so that bit will always be 0 for user-space addresses meaning bits 63:48 must also be 0 to be in canonical form.
RISC-V has essentially the same restriction (see 4.5.1 in the RISC-V privileged specification) "instruction fetch addresses and load and store effective addresses, which are 64 bits, must have bits 63-48 all equal to bit 47, or else a page-fault exception will occur." The virtual memory layout used by the Linux kernel uses the same approach as for x86-64.
AArch64 has a slight variant on the above which essentially provides a 49-bit address space (meaning user-space virtual memory can cover 256TiB rather than 128TiB). As described in the Armv8-A address translation documentation (section 3), for a 48-bit address space bits 63:48 must be all 0s or all 1s. However, they don't need to be a copy of bit 47, and a different address translation table is used depending on whether bits 63:48 are 1 or 0. This allows splitting kernel/user addresses without giving up bit 47.

What if virtual addresses are wider than 48 bits?

So we've covered the easy case, where you can freely (ab)use the upper 16 bits for your own purposes. But what if you're running on a system that has wider than 48 bit virtual addresses? How do you know that's the case? And is it possible to limit virtual addresses to the 48-bit range if you're sure you don't need the extra bits?

You can query the virtual address width from the command-line by catting /proc/cpuinfo, which might include a line like address sizes : 39 bits physical, 48 bits virtual. I'd hope there's a way to get the same information without parsing /proc/cpuinfo, but I haven't been able to find it.

As for how to keep using those upper bits on a system with wider virtual addresses, helpfully the behaviour of mmap is defined with this compatibility in mind. It's explicitly documented for x86-64, for AArch64 and for RISC-V that addresses beyond 48-bits won't be returned unless a hint parameter beyond a certain width is used (the details are slightly different for each target). This means if you're confident that nothing within your process is going to be passing such hints to mmap (including e.g. your malloc implementation), or at least that you'll never need to try to reuse upper bits of addresses produced in this way, then you're free to presume the system uses no more than 48 bits of virtual address.

Top byte ignore and similar features

Up to this point I've completely skipped over the various architectural features that allow some of the upper bits to be ignored upon dereference, essentially providing hardware support for this type of storage of additional data within pointeres by making additional masking unnecessary.

x86-64 keeps things interesting by having slightly different variants of this for Intel and AMD.
- Intel introduced Linear Address Masking (LAM), documented in chapter 6 of their document on instruction set extensions and future features. If enabled this modifies the canonicality check so that, for instance, on a system with 48-bit virtual addresses bit 47 must be equal to bit 63. This would allow bits 62:48 (15 bits) can be freely used with no masking needed. "LAM57" allows 62:57 to be used (6 bits). It seems as if Linux is currently opting to only support LAM57 and not LAM48. Support for LAM can be configured separately for user and supervisor mode, but I'll refer you to the Intel docs for details.
- AMD instead describes Upper Address Ignore (see section 5.10) which allows bits 63:57 (7 bits) to be used, and unlike LAM doesn't require bit 63 to match the upper bit of the virtual address. As documented in LWN, this caused some concern from the Linux kernel community. Unless I'm missing it, there doesn't seem to be any level of support merged in the Linux kernel at the time of writing.
RISC-V has the proposed pointer masking extension which defines new supervisor-level extensions Ssnpm, Smnpm, and Smmpm to control it. These allow PMLEN to potentially be set to 7 (masking the upper 7 bits) or 16 (masking the upper 16 bits). In usual RISC-V style, it's not mandated which of these are supported, but the draft RVA23 profile mandates that PMLEN=7 must be supported at a minimum. Eagle-eyed readers will note that the proposed approach has the same issue that caused concern with AMD's Upper Address Ignore, namely that the most significant bit is no longer required to be the same as the top bit of the virtual address. This is noted in the spec, with the suggestion that this is solvable at the ABI level and some operating systems may choose to mandate that the MSB not be used for tagging.
AArch64 has the Top Byte Ignore (TBI) feature, which as the name suggests just means that the top 8 bits of a virtual address are ignored when used for memory accesses and can be used to store data. Any other bits between the virtual address width and top byte must be set to all 0s or all 1s, as before. TBI is also used by Arm's Memory Tagging Extension (MTE), which uses 4 of those bits as the "key" to be compared against the "lock" tag bits associated with a memory location being accessed. Armv8.3 defines another potential consumer of otherwise unused address bits, pointer authentication which uses 11 to 31 bits depending on the virtual address width if TBI isn't being used, or 3 to 23 bits if it is.

A relevant historical note that multiple people pointed out: the original Motorola 68000 had a 24-bit address bus and so the top byte was simply ignored which caused well documented porting issues when trying to expand the address space.

Storing data in least significant bits

Another commonly used trick I'd be remiss not to mention is repurposing a small number of the least significant bits in a pointer. If you know a certain set of pointers will only ever be used to point to memory with a given minimal alignment, you can exploit the fact that the lower bits corresponding to that alignment will always be zero and store your own data there. As before, you'll need to account for the bits you repurpose when accessing the pointer - in this case either by masking, or by adjusting the offset used to access the address (if those least significant bits are known).

As suggested by Per Vognsen, after this article was first published, you can exploit x86's scaled index addressing mode to use up to 3 bits that are unused due to alignment, but storing your data in the upper bits. The scaled index addressing mode meaning there's no need for separate pointer manipulation upon access. e.g. for an 8-byte aligned address, store it right-shifted by 3 and use the top 3 bits for metadata, then scaling by 8 using SIB when accessing (which effectively ignores the top 3 bits). This has some trade-offs, but is such a neat trick I felt I have to include it!

Some real-world examples

To state what I hope is obvious, this is far from an exhaustive list. The point of this quick blog post was really to discuss cases where additional data is stored alongside a pointer, but of course unused bits can also be exploited to allow a more efficient tagged union representation (and this is arguably more common), so I've included some examples of that below:

The fixie trie, is a variant of the trie that uses 16 bits in each pointer to store a bitmap used as part of the lookup logic. It also exploits the minimum alignment of pointers to repurpose the least significant bit to indicate if a value is a branch or a leaf.
On the topic of storing data in the least significant bits, we have a handy PointerIntPair class in LLVM to allow the easy implementation of this optimisation. There's also an 'alignment niches' proposal for Rust which would allow this kind of optimisation to be done automatically for enums (tagged unions). Another example of repurposing the LSB found in the wild would be the Linux kernel using it for a spin lock (thanks Vegard Nossum for the tip, who notes this is used in the kernel's directory entry cache hashtable). There are surely many many more examples.
Go repurposes both upper and lower bits in its taggedPointer, used internally in its runtime implementation.
If you have complete control over your heap then there's more you can do to make use of embedded metadata, including using additional bits by avoiding allocation outside of a certain range and using redundant mappings to avoid or reduce the need for masking. OpenJDK's ZGC is a good example of this, utilising a 42-bit address space for objects and upon allocation mapping pages to different aliases to allow pointers using their metadata bits to be dereferenced without masking.
A fairly common trick in language runtimes is to exploit the fact that values can be stored inside the payload of double floating point NaN (not a number) values and overlap it with pointers (knowing that the full 64 bits aren't needed) and even small integers. There's a nice description of this in JavaScriptCore, but it was famously used in LuaJIT. Andy Wingo also has a helpful write-up. Along similar lines, OCaml steals just the least significant bit in order to efficiently support unboxed integers (meaning integers are 63-bit on 64-bit platforms and 31-bit on 32-bit platforms).
Apple's Objective-C implementation makes heavy use of unused pointer bits, with some examples documented in detail on Mike Ash's excellent blog (with a more recent scheme described on Brian T. Kelley's blog. Inlining the reference count (falling back to a hash lookup upon overflow) is a fun one. Another example of using the LSB to store small strings in-line is squoze.
V8 opts to limit the heap used for V8 objects to 4GiB using pointer compression, where an offset is used alongside the 32-bit value (which itself might be a pointer or a 31-bit integer, depending on the least significant bit) to refer to the memory location.
As this list is becoming more of a collection of things slightly outside the scope of this article I might as well round it off with the XOR linked list, which reduces the storage requirements for doubly linked lists by exploiting the reversibility of the XOR operation.
I've focused on storing data in conventional pointers on current commodity architectures but there is of course a huge wealth of work involving tagged memory (also an area where I've dabbled - something for a future blog post perhaps) and/or alternative pointer representations. I've touched on this with MTE (mentioned due to its interaction with TBI), but another prominent example is of course CHERI which moves to using 128-bit capabilities in order to fit in additional inline metadata. David Chisnall provided some observations based on porting code to CHERI that relies on the kind of tricks described in this post.

Fin

What did I miss? What did I get wrong? Let me know on Mastodon or email (asb@asbradbury.org).

You might be interested in the discussion of this article on lobste.rs, on HN, on various subreddits, or on Mastodon.

Article changelog

2023-12-02:
- Reference Brian T. Kelley's blog providing a more up-to-date description of "pointer tagging" in Objective-C. Spotted on Mastodon.
2023-11-27:
- Mention Squoze (thanks to Job van der Zwan).
- Reworded the intro so as not to claim "it's quite well known" that the maximum virtual address width is typically less than 64 bits. This might be interpreted as shaming readers for not being aware of that, which wasn't my intent. Thanks to HN reader jonasmerlin for pointing this out.
- Mention CHERI is the list of "real world examples" which is becoming dominated by instances of things somewhat different to what I was describing! Thanks to Paul Butcher for the suggestion.
- Link to relevant posts on Mike Ash's blog (suggested by Jens Alfke).
- Link to the various places this article is being discussed.
- Add link to M68k article suggested by Christer Ericson (with multiple others suggesting something similar - thanks!).
2023-11-26:
- Minor typo fixes and rewordings.
- Note the Linux kernel repurposing the LSB as a spin lock (thanks to Vegard Nossum for the suggestion).
- Add SIB addressing idea shared by Per Vognsen.
- Integrate note suggested by Andy Wingo that explicit masking often isn't needed when the least significant pointer bits are repurposed.
- Add a reference to Arm Pointer Authentication (thanks to the suggestion from Tzvetan Mikov).
2023-11-26: Initial publication date.

What's new for RISC-V in LLVM 17

2023-10-10T12:00:00Z

LLVM 17 was released in the past few weeks, and I'm continuing the tradition of writing up some selective highlights of what's new as far as RISC-V is concerned in this release. If you want more general, regular updates on what's going on in LLVM you should of course subscribe to my newsletter.

In case you're not familiar with LLVM's release schedule, it's worth noting that there are two major LLVM releases a year (i.e. one roughly every 6 months) and these are timed releases as opposed to being cut when a pre-agreed set of feature targets have been met. We're very fortunate to benefit from an active and growing set of contributors working on RISC-V support in LLVM projects, who are responsible for the work I describe below - thank you! I coordinate biweekly sync-up calls for RISC-V LLVM contributors, so if you're working in this area please consider dropping in.

Code size reduction extensions

A family of extensions referred to as the RISC-V code size reduction extensions was ratified earlier this year. One aspect of this is providing ways of referring to subsets of the standard compressed 'C' (16-bit instructions) extension that don't include floating point loads/stores, as well as other variants. But the more meaningful additions are the Zcmp and Zcmt extensions, in both cases targeted at embedded rather than application cores, reusing encodings for double-precision FP store.

Zcmp provides instructions that implement common stack frame manipulation operations that would typically require a sequence of instructions, as well as instructions for moving pairs of registers. The RISCVMoveMerger pass performs the necessary peephole optimisation to produce cm.mva01s or cm.mvsa01 instructions for moving to/from registers a0-a1 and s0-s7 when possible. It iterates over generated machine instructions, looking for pairs of c.mv instructions that can be replaced. cm.push and cm.pop instructions are generated by appropriate modifications to the RISC-V function frame lowering code, while the RISCVPushPopOptimizer pass looks for opportunities to convert a cm.pop into a cm.popretz (pop registers, deallocate stack frame, and return zero) or cm.popret (pop registers, deallocate stack frame, and return).

Zcmt provides the cm.jt and cm.jalt instructions to reduce code size needed for implemented a jump table. Although support is present in the assembler, the patch to modify the linker to select these instructions is still under review so we can hope to see full support in LLVM 18.

The RISC-V code size reduction working group have estimates of the code size impact of these extensions produced using this analysis script. I'm not aware of whether a comparison has been made to the real-world results of implementing support for the extensions in LLVM, but that would certainly be interesting.

Vectorization

LLVM has two forms of auto-vectorization, the loop vectorizer and the SLP (superword-level parallelism) vectorizer. The loop vectorizer was enabled during the LLVM 16 development cycle, while the SLP vectorizer was enabled for this release. Beyond that, there's been a huge number of incremental improvements for vector codegen such that isn't always easy to pick out particular highlights. But to pick a small set of changes:

Version 0.12 of the RISC-V vector C intrinsics specification is now supported by Clang. As noted in the release notes, the hope is there will not be new incompatibilities introduced prior to v1.0.
There were lots of minor codegen improvements, one example would be improvements to the RISCVInsertVSETVLI pass to avoid additional unnecessary insertions vsetivli instruction that is used to modify the vtype control register.
It's not particularly user visible, but there was a lot of refactoring of vector pseudoinstructions used internally during instruction selection (following this thread. The added documentation will likely be helpful if you're hoping to better understand this.
You might be aware that LMUL in the RISC-V vector extension controls grouping of vector registers, for instance rather than 32 vector registers, you might want to set LMUL=4 to treat them as 8 registers that are 4 times as large. The "best" LMUL is going to vary depending on both the target microarchitecture and factors such as register pressure, but a change was made so LMUL=2 is the new default.
llvm-mca (the LLVM Machine Code Analyzer) is a performance analysis tool that uses information such as LLVM scheduling models to statically estimate the performance of machine code on a specific CPU. There were at least two changes relevant to llvm-mca and RISC-V vector support: scheduling information for RVV on SiFive7 cores (which of course is used outside of llvm-mca as well), and support for vsetivli/vsetvli 'instruments'. llvm-mca has the concept of an 'instrument region', a section of assembly with an LLVM-MCA comment that can (for instance) indicate the value of a control register that would affect scheduling. This can be used to set LMUL (register grouping) for RISC-V, however in the case of the immediate forms of vsetvl occuring in the input, LMUL can be statically determined.

If you want to find out more about RISC-V vector support in LLVM, be sure to check out my Igalia colleague Luke Lau's talk at the LLVM Dev Meeting this week (I'll update this article when slides+recording are available).

Other ISA extensions

It wouldn't be a RISC-V article without a list of hard to interpret strings that claim to be ISA extension names (Zvfbfwma is a real extension, I promise!). In addition to the code size reduction extension listed above there's been lots of newly added or updated extensions in this release cycle. Do refer to the RISCVUsage documentation for something that aims to be a complete list of what is supported (occasionally there are omissions) as well as clarity on what we mean by an extension being marked as "experimental".

Here's a partial list:

Code generation support for the Zfinx, Zdinx, Zhinx, and Zhinxmin extensions. These extensions provide support for single, double, and half precision floating point instructions respectively, but define them to operate on the general purpose register file rather than requiring an additional floating point register file. This reduces implementation cost on simple core designs.
Support for a whole range of vendor-defined extensions. e.g. XTHeadBa (address gneeration), XTheadBb (basic bit manipulation), Xsfvcp (SiFive VCIX), XCVbitmanip (CORE-V bit manipulation custom instructions) and many more (see the release notes.
Experimental vector crypto extension support was updated to version 0.5.1 of the specification.
Experimental support was added for version 0.2 of the Zfa extension (providing additional floating-point instructions).
Assembler/disassembler support for an experimental family of extensions to support operations on the bfloat16 floating-point format. Zfbfmin, Zvfbfmin, and Zvfbfwma.
Assembler/disassembler support for the experimental Zacas extension (atomic compare-and-swap).

It landed after the 17.x branch so isn't in this release, but in the future you'll be able to use --print-supported-extensions with Clang to have it print a table of supported ISA extensions (the same flag has now been implemented for Arm and AArch64 too).

Other additions and improvements

As always, it's not possible to go into detail on every change. A selection of other changes that I'm not able to delve into more detail on:

Initial RISC-V support was added to LLVM's BOLT post-link optimizer and various fixes / feature additions made to JITLink, thanks to the work of my Igalia colleague Job Noorman. There's actually a lot to say about this work, but I don't need to because Job has written up and excellent blog post on it that I highly encourage you go and read.
LLD gained support for some of the relaxations involving the global pointer.
I expect there'll be more to say about this in future releases, but there's been incremental progress on RISC-V GlobalISel in the LLVM 17 development cycle (which has continued after). You might be interested in the slides from my GlobalISel by example talk at EuroLLVM this year. Ivan Baev at SiFive is also set to speak about some of this work at the RISC-V Summit in November.
Clang supports a form of control-flow integrity called KCFI. This is used by low-level software like the Linux kernel (see CONFIG_CFI_CLANG in the Linux tree) but the target-specific parts were previously unimplemented for RISC-V. This gap was filled for the LLVM 17 release.
LLVM has its own work-in-progress libc implementation, and the RISC-V implementations of memcmp, bcmp, memset, and memcpy all gained optimised RISC-V specific versions. There will of course be further updates for LLVM 18, including the work from my colleague Mikhail R Gadelha on 32-bit RISC-V support.

Apologies if I've missed your favourite new feature or improvement - the LLVM release notes will include some things I haven't had space for here. Thanks again for everyone who has been contributing to make the RISC-V in LLVM even better.

If you have a RISC-V project you think me and my colleagues and at Igalia may be able to help with, then do get in touch regarding our services.

2023Q2 week log

2023-04-10T12:00:00Z

I tend to keep quite a lot of notes on the development related (sometimes at work, sometimes not) I do on a week-by-week basis, and thought it might be fun to write up the parts that were public. This may or may not be of wider interest, but it aims to be a useful aide-mémoire for my purposes at least. Weeks with few entries might be due to focusing on downstream work (or perhaps just a less productive week - I am only human!).

Week of 29th May 2023

Posted D151663, implementing support for bf16 truncate/extend of hard FP targets.
Responded to user query about gating of CSRs.
Filed issue about shift and right-click and xfce4-terminal (after migrating to it due to frustration with gnome-terminal not supporting setting the urgent hint upon receiving a terminal bell).
Spread the word about my keynote about LLVM next week at the RISC-V Summit Europe.
A few reviews on RISC-V psABI or ASM manual PRs (e.g. atomics ABI, floating point in the asm manual).
Less activity this week due to being on holiday.
LLVM Weekly #491.

Week of 22nd May 2023

Posted a patch to generalise the shouldExtendTypeInLibcall hook so it applies to half and bfloat16.
Updated bfloat16 psABI PR, which has now been merged.
Posted D151363 a patch to implement soft FP legalisation for bf16 FP_EXTEND and BF16_TO_FP after abandoning patch to add an extenbfsf2 libcall (which would match libgcc, but add no real value).
Identified a bug in the ABI used for half FP libcalls and posted a patch to fix it.
Some misc small cleanups like making zfbfmin imply the F extension, cleaning up bfloat16 tests (1, 2).
Prepared agenda for an ran biweekly RISC-V LLVM sync-up call.
LLVM Weekly #490.

Week of 15th May 2023

Corrected Clang codegen support for half FP types when the zhinx extension is available (D150777.
Rebased and committed patches to implement MC layer support for the bfloat16 extensions (unblocked now a new PDF was posted in the riscv-bfloat16 repo). Zfbfmin, zvfbfmin, zvfbfwma. Also made a trivial typo fix to the spec.
Looked at cleaning up the usage of report_fatal_error in the RISC-V backend and also fixed a bug encountered while looking at this. D150674.
Participated in the ongoing discussion about adding atomics lowering to the RISC-V psABI, including a slightly altered lowering of some primitives in order to allow for forwards compatibility with "table A.7".
Usual mix of upstream LLVM reviews, and a number of RISC-V psABI or ASM manual reviews.
LLVM Weekly #489.
Missed some weeks - busy with EuroLLVM etc.

Week of 17th April 2023

Still pinging for an updated riscv-bfloat1y spec version that incorporates the fcvt.bf16.s encoding fix.
Bumped the version of the experimental Zfa RISC-V extension supported by LLVM to 0.2 (D146834). This was very straightforward as after inspecting the spec history, it was clear there were no changes that would impact the compiler.
Filed a couple of pull requests against the riscv-zacas repo (RISC-V Atomic Compare and Swap extension).
- #8 made the dependency on the A extension explicit.
- #7 attempted to explicitly reference the extension for misaligned atomics, though it seems won't be merged. I do feel uncomfortable with RISC-V extensions that can have their semantics changed by other standard extensions without this possibility being called out very explicitly. As I note in the PR, failure to appreciate this might mean that conformance tests written for zacas might fail on a system with zacas_zam. I see a slight parallel to a recent discussion about RISC-V profiles.
Fixed the canonical ordering used for ISA naming strings in RISCVISAInfo (this will mainly affect the string stored in build attributes). This was fixed in D148615 which built on the pre-committed test case.
A whole bunch of upstream LLVM reviews. As noted in D148315 I'm thinking we should probably relaxing the ordering rules for ISA strings in -march in order to avoid issues due to spec changes and incompatibilities between GCC and Clang.
LLVM Weekly #485.

Week of 10th April 2023

Some days off due to the Easter holidays, so less to report this week.
Updated RISC-V bfloat16 patches (Zfbfmin, Zvfbfmin, Zvfbfwma), incorporating new fcvt.bf16.s encoding. Also filed an issue about the way in which the dependencies of the vector bfloat16 extensions is specified.
Blogged about updating the Wren language benchmarks.
Variety of upstream LLVM reviews.
LLVM Weekly #484.

Week of 3rd April 2023

Some days off due to the Easter holidays, so less to report this week.
Posted MC layer (assembler/disassembler) patches for the bfloat16 extensions: Zfbfmin, Zvfbfmin, Zvfbfwma.
- Also posted a PR to the riscv-bfloat16 spec to clarify the vector extension dependencies.
- Pinged on my bug report about the fcvt.bf16.s encoding clash. Once this is resolved, the LLVM MC layer patches can land.
Updated authorship information for riscv-toolchain-conventions which now has a range of contributors beyond myself.
Usual mix of upstream LLVM reviews. This included some discussion on changing the shadow call stack register to x3 which spilled into the psABI PR where I suggested some alternate wording.
LLVM Weekly #483.

Article changelog

2023-06-05: Added notes for the week of 22nd May 2023 and week fo 29th May 2023.
2023-05-22: Added notes for the week of 15th May 2023.
2023-04-24: Added notes for the week of 17th April 2023.
2023-04-17: Added notes for the week of 10th April 2023.
2023-04-10: Initial publication date.

Updating Wren's benchmarks

2023-04-10T12:00:00Z

Wren is a "small, fast, class-based, concurrent scripting language", originally designed by Bob Nystrom (who you might recognise as the author of Game Programming Patterns and Crafting Interpreters. It's a really fun language to study - the implementation is compact and easily readable, and although class-based languages aren't considered very hip these days there's a real elegance to its design. I saw Wren's performance page hadn't been updated for a very long time, and especially given the recent upstream interpreter performance work on Python, was interested in seeing how performance on these microbencharks has changed. Hence this quick post to share some new numbers.

New results

To cut to the chase, here are the results I get running the same set of benchmarks across a collection of Python, Ruby, and Lua versions (those available in current Arch Linux).

Method Call:

wren0.4	0.079s
luajit2.1 -joff	0.090s
ruby2.7	0.102s
ruby3.0	0.104s
lua5.4	0.123s
lua5.3	0.156s
python3.11	0.170s
lua5.2	0.184s
mruby	0.193s
python3.10	0.313s

Delta Blue:

wren0.4	0.086s
python3.11	0.106s
python3.10	0.202s

Binary Trees:

luajit2.1 -joff	0.073s
ruby2.7	0.113s
ruby3.0	0.115s
python3.11	0.137s
lua5.4	0.138s
wren0.4	0.144s
mruby	0.163s
python3.10	0.186s
lua5.3	0.195s
lua5.2	0.196s

Recursive Fibonacci:

luajit2.1 -joff	0.055s
lua5.4	0.090s
ruby2.7	0.109s
ruby3.0	0.117s
lua5.3	0.126s
lua5.2	0.138s
wren0.4	0.148s
python3.11	0.157s
mruby	0.185s
python3.10	0.252s

I've used essentially the same presentation and methodology as in the original benchmark, partly to save time pondering the optimal approach, partly so I can redirect any critiques to the original author (sorry Bob!). Benchmarks do not measure interpreter startup time, and each benchmark is run ten times with the median used (thermal throttling could potentially mean this isn't the best methodology, but changing the number of test repetitions to e.g. 1000 seems to have little effect).

The tests were run on a machine with an AMD Ryzen 9 5950X processor. wren 0.4 as of commit c2a75f1 was used as well as the following Arch Linux packages:

lua52-5.2.4-5
lua53-5.3.6-1
lua-5.4.4-3,
luajit-2.1.0.beta3.r471.g505e2c03-1
mruby-3.1.0-1
python-3.10.10-1
python-3.11.3-1 (taken from Arch Linux's staging repo)
ruby2.7-2.7.7-1
ruby-3.0.5-1

The Python 3.10 and 3.11 packages were compiled with the same GCC version (12.2.1 according to python -VV), though this won't necessarily be true for all other packages (e.g. the lua52 and lua53 packages are several years old so will have been built an older GCC).

I've submitted a pull request to update the Wren performance page.

Old results

The following results are copied from the Wren performance page (archive.org link ease of comparison. They were run on a MacBook Pro 2.3GHz Intel Core i7 with Lua 5.2.3, LuaJIT 2.0.2, Python 2.7.5, Python 3.3.4, ruby 2.0.0p247.

Method Call:

wren2015	0.12s
luajit2.0 -joff	0.16s
ruby2.0	0.20s
lua5.2	0.35s
python3.3	0.78s
python2.7	0.85s

DeltaBlue:

wren2015	0.13s
python3.3	0.48s
python2.7	0.57s

Binary Trees:

luajit2.0 -joff	0.11s
wren2015	0.22s
ruby2.0	0.24s
python2.7	0.37s
python3.3	0.38s
lua5.2	0.52s

Recursive Fibonacci:

luajit2.0 -joff	0.10s
wren2015	0.20s
ruby2.0	0.22s
lua5.2	0.28s
python2.7	0.51s
python3.3	0.57s

Observations

A few takeaways:

LuaJIT's bytecode interpreter remains incredibly fast (though see this blog post for a methodology to produce an even faster interpreter).
The performance improvements in Python 3.11 were well documented and are very visible on this set of microbenchparks.
I was more surprised by the performance jump with Lua 5.4, especially as the release notes give few hints of performance improvements that would be reflected in these microbenchmarks. The LWN article about the Lua 5.4 release however did note improved performance on a range of benchmarks.
Wren remains speedy (for these workloads at least), but engineering work on other interpreters has narrowed that gap for some of these benchmarks.
I haven't taken the time to compare the January 2015 version of Wren used for the benchmarks vs present-day Wren 0.4. It would be interesting to explore that though.
A tiny number of microbenchmarks have been used in this performance test. It wouldn't be wise to draw general conclusions - this is just a bit of fun.

Appendix: Benchmark script

Health warning: this is incredibly quick and dirty (especially the repeated switching between the python packages to allow testing both 3.10 and 3.11):

#!/usr/bin/env python3

# Copyright Muxup contributors.
# Distributed under the terms of the MIT license, see LICENSE for details.
# SPDX-License-Identifier: MIT

import statistics
import subprocess

out = open("out.md", "w", encoding="utf-8")


def run_single_bench(bench_name, bench_file, runner_name):
    bench_file = "./test/benchmark/" + bench_file
    if runner_name == "lua5.2":
        bench_file += ".lua"
        cmdline = ["lua5.2", bench_file]
    elif runner_name == "lua5.3":
        bench_file += ".lua"
        cmdline = ["lua5.3", bench_file]
    elif runner_name == "lua5.4":
        bench_file += ".lua"
        cmdline = ["lua5.4", bench_file]
    elif runner_name == "luajit2.1 -joff":
        bench_file += ".lua"
        cmdline = ["luajit", "-joff", bench_file]
    elif runner_name == "mruby":
        bench_file += ".rb"
        cmdline = ["mruby", bench_file]
    elif runner_name == "python3.10":
        bench_file += ".py"
        subprocess.run(
            [
                "sudo",
                "pacman",
                "-U",
                "--noconfirm",
                "/var/cache/pacman/pkg/python-3.10.10-1-x86_64.pkg.tar.zst",
            ],
            check=True,
        )
        cmdline = ["python", bench_file]
    elif runner_name == "python3.11":
        bench_file += ".py"
        subprocess.run(
            [
                "sudo",
                "pacman",
                "-U",
                "--noconfirm",
                "/var/cache/pacman/pkg/python-3.11.3-1-x86_64.pkg.tar.zst",
            ],
            check=True,
        )
        cmdline = ["python", bench_file]
    elif runner_name == "ruby2.7":
        bench_file += ".rb"
        cmdline = ["ruby-2.7", bench_file]
    elif runner_name == "ruby3.0":
        bench_file += ".rb"
        cmdline = ["ruby", bench_file]
    elif runner_name == "wren0.4":
        bench_file += ".wren"
        cmdline = ["./bin/wren_test", bench_file]
    else:
        raise SystemExit("Unrecognised runner")

    times = []
    for _ in range(10):
        bench_out = subprocess.run(
            cmdline, capture_output=True, check=True, encoding="utf-8"
        ).stdout
        times.append(float(bench_out.split(": ")[-1].strip()))
    return statistics.median(times)


def do_bench(name, file_base, runners):
    results = {}
    for runner in runners:
        results[runner] = run_single_bench(name, file_base, runner)
    results = dict(sorted(results.items(), key=lambda kv: kv[1]))
    longest_result = max(results.values())
    out.write(f"**{name}**:\n")
    out.write('<table class="chart">\n')
    for runner, result in results.items():
        percent = round((result / longest_result) * 100)
        out.write(
            f"""\
  <tr>
    <th>{runner}</th><td><div class="chart-bar" style="width: {percent}%;">{result:.3f}s&nbsp;</div></td>
  </tr>\n"""
        )
    out.write("</table>\n\n")


all_runners = [
    "lua5.2",
    "lua5.3",
    "lua5.4",
    "luajit2.1 -joff",
    "mruby",
    "python3.10",
    "python3.11",
    "ruby2.7",
    "ruby3.0",
    "wren0.4",
]
do_bench("Method Call", "method_call", all_runners)
do_bench("Delta Blue", "delta_blue", ["python3.10", "python3.11", "wren0.4"])
do_bench("Binary Trees", "binary_trees", all_runners)
do_bench("Recursive Fibonacci", "fib", all_runners)
print("Output written to out.md")

2023Q1 week log

2023-02-27T12:00:00Z

Week of 27th March 2023

Submitted a backport request to 16.0.1 for my recent fixes to llvm-objdump (and related tools) when encountering unrecognised RISC-V base or ISA extension versions, or unrecognised ISA extension names.
Landed a tweak to the RISC-V ISA manual to make it clear that HINT encodings aren't "reserved" in terms of being part of the defined "reserved instruction-set category". Thanks to Andrew Waterman for suggesting a simpler fix than my first attempt.
I'm now on Mastodon, at @asb@fosstodon.org and @llvmweekly@fosstodon.org.
Proposed alternate wording for the RISC-V LLVM doc updates reflecting recent ISA versioning discussions.
Set agenda for and ran the usual biweekly RISC-V LLVM contributor sync up call.
Added the Renesas R9A06G150 to my commercially available RISC-V silicon list.
Posted and landed patches to implement MC layer and codegen support for the experimental Zicond (integer conditional operations) extension (D146946, D147147). This is essentially the same as the XVentanaCondOps extension.
Advertised the ongoing discussion about changing the shadow call stack register on RISC-V through an RFC.
It was pointed out to me that the expansion of seq_cst atomic ops to RISC-V lr/sc loops was slightly stronger than required by the mapping table in the ISA manual. Specifically, sc.{w|d}.rl is sufficient rather than sc.{w|d}.aqrl. Fixed with D146933.
Filed issue about the fcvt.bf16.s instruction encoding colliding with fround.h (instructions from zfbfmin and zfa respectively).
Usual mix of upstream LLVM reviews. We were now able to bump the versions of the standard ISA extensions LLVM claims to support. As noted in my previous RFC, LLVM was reporting the wrong version information for the A/F/D extensions.
LLVM Weekly #482.

Week of 20th March 2023

Landed patches to fix RISC-V ISA extension versioning related issues in llvm-objdump and related tools (D146070, D146113, and D146114). Also patches to fix an ABI bug with _Float16 lowering on RISC-V (D142326, D145074).
Opened a few issues on the aichat (command line ChatGPT client) repo: one for maximum line width, another for word wrapping, and a suggestion on converting one-shot questions to conversations.
Added the HPM6750 to my commercially available RISC-V silicon list.
My tutorial and lightning talk proposals were accepted for EuroLLVM!
Some bits and pieces related to the RISC-V bfloat16 spec and also Zfh.
- Drafted a Zfbfinxmin extension definition (primarily for symmetry with the existing z*inx[min] extensions.
- Fixing a missed predicate for PseudoQuietFCMP.
- Minor clarification to the riscv-bfloat16 spec.
Usual mix of upstream LLVM reviews.
LLVM Weekly #481.

Week of 13th March 2023

Most importantly, added some more footer images for this site from the Quick Draw dataset. Thanks to my son (Archie, 5) for the assistance.
Reviewed submissions for EuroLLVM (I'm on the program committee).
Added note to the commercially available RISC-V silicon post about a hardware bug in the Renesas RZ/Five.
Finished writing and published what's new for RISC-V in LLVM 16 article and took part in some of the discussions in the HN and Reddit threads (it's on lobste.rs too, but that didn't generate any comments).
Investigated an issue where inline asm with the m constraint was generating worse code on LLVM vs GCC, finding that LLVM conservatively lowers this to a single register, while GCC treats m as reg+imm, relying on users indicating A when using a memory operand with an instruction that can't take an immediate offset. Worked with a colleague who posted D146245 to fix this.
Set agenda for and ran the biweekly RISC-V LLVM contributor sync call as usual.
Bisected reported LLVM bug #61412, which as it happens was fixed that evening by D145474 being committed. We hope to backport this to 16.0.1.
Did some digging on a regression (compiler crash) for -Oz, bisecting it to the commit that enabled machine copy propagation by default. I found the issue was due to machine copy propagation running after the machine outliner, and incorrectly determining that some register writes in outlined functions were not live-out. I posted and landed D146037 to fix this by running machine copy propagation earlier in the pipeline, though a more principled fix would be desirable.
Filed a PR against the riscv-isa-manual to disambiguate the use of the term "reserved" for HINT instructions. I've also been looking at the proposed bfloat16 extension recently and filed an issue to clarify if Zfbfinxmin will be defined (as all the other floating point extensions so far have an *inx twin.
Almost finished work to resolve issues related to overzealous error checking on RISC-V ISA naming strings (with llvm-objdump and related tools being the final piece).
- Landed D145879 and D145882 to expand RISCVISAInfo test coverage and fix an issue that surfaced through that.
- Posted a pair of patches that makes llvm-objdump and related tools tolerant of unrecognised versions of ISA extensions. D146070 resolves this for the base ISA in a minimally invasive way, while D146114 solves this for other extensions, moving the parsing logic to using the parseNormalizedArchString function I introduced to fix a similar issue in LLD. This built on some directly committed work to expand testing.
The usual assortment of upstream LLVM reviews.
LLVM Weekly #480.

Week of 6th March 2023

Had a really useful meeting with a breakout group from the RISC-V LLVM sync-up calls about the long-standing issues related to ISA extension versioning, error handling for this, and other related issues.
- Related to this, posted D145879 and D145882 to flesh out testing of RISCVISAInfo::parseArchString ahead of further improvements, and to start to fix identified issues.
Incorporated more clarifications submitted to me about the commercially available RISC-V SoCs post, particularly around SiFive cores.
Shared some thoughts on attracting more people to the LLVM Foundation strategic planning sessions.
Posted and committed an LLVM patch to migrate the RISC-V backend to using shared MCELFStreamer code for attribute emission. The initial implementation was largely derived from Arm's version of the same feature but a later refactoring managed to move this logic to common code, which we can now reuse.
Some small tasks related to the RISC-V LLVM build-bot: trying to find a path forwards for a simple patch to enable RISC-V support in libcxx, and clarifying how often the staging buildmaster configuration is updated.
Posted a docs patch to clarify Clang's -fexceptions in follow-up to discussion in issue 61216.
Did some final preparations for the LLVM 16.0.0 release - committing some cleaned up release notes.
A variety of upstream LLVM reviews, and left some thoughts on using sub-modules for RVV intrinsics tests.
LLVM Weekly #479.

Week of 27th February 2023

Completed (to the point I was happy to publish at least) my attempt to enumerate the commercially available RISC-V SoCs. I'm very grateful to have received a whole range of suggested additions and clarifications over the weekend, which have all been incorporated.
Ran the usual biweekly RISC-V LLVM sync-up call. Topics included outstanding issues for LLVM 16.x (no major issues now my backport request to fix and LLD regression was merged), an overview off _Float16 ABI lowering fixes, GP relaxation in LLD, my recent RISC-V buildbot, and some vectorisation related issues.
Investigated and largely resolved a issues related to ABI lowering of _Float16 for RISC-V. Primarily, we weren't handling the cases where a GPR+FPR or a pair of FPRs are used to pass small structs including _Float16.
- Part of this work involved rebasing my previous patches to refactor our RISC-V ABI lowering tests in Clang. Now that a version of my improvements to update_cc_test_check.py --function-signature (required for the refactor) landed as part of D144963, this can hopefully be completed.
- Committed a number of simple test improvements related to half floats. e.g. 570995e, 81979c3, 34b412d.
- Posted D145070 to add proper coverage for _Float16 ABI lowering, and D145074 to fix it. Also D145071 to set the HasLegalHalfType property, but the semantics of that are less clear.
- Posted a strawman psABI patch for __bf16, needed for the RISC-V bfloat16 extension.
Attended the Cambridge RISC-V Meetup.
After seeing the Helix editor discussed on lobste.rs, retried my previously shared large Markdown file test case. Unfortunately it's still unusably slow to edit, seemingly due to a tree-sitter related issue.
Cleaned up the static site generator used for this site a bit. e.g. now my fixes (#157, #158, #159) for the traverse() helper in mistletoe where merged upstream, I removed my downstream version.
The usual mix of upstream LLVM reviews.
Had a day off for my birthday.
Publicly shared this week log for the first time.
LLVM Weekly #478.

Week of 20th February 2023

Iterated on D144353 (aiming to fix LLD regression related to merging RISC-V attributes) based on review feedback and committed it.
- Created an issue to track this as a regression, aiming for a backport into 16.0.0, and requested that backport.
- Also some related discussion in ClangBuiltLinux issues #1777 and #1808.
Committed my llvm-zorg patch to add the qemu-user based RISC-V builder, after finalising provisioning the machine to run it. The builder is live on the LLVM staging buildmaster as clang-rv64gc-qemu-user-single-stage.
- Worked to resolve remaining test failures and stability issues. One recurrent issue was an assert in ___pthread_mutex_lock when executing ccache. Setting inode_cache=false in the local ccache config seems to avoid this.
- Posted a couple of patches - D144464 and D144465 to tweak the LLVM docs on setting up a builder, based on my experience doing so.
Chased for reviews and clarification about pre-commit test requirements for my libcxx RISC-V test fix patch, D134158.
Committed a couple of further test updates for Wasm in LLVM ahead of some upcoming patches. 771261f 1ae8597.
Left some quick notes on the LLVM RFC shepherds proposal.
A variety of upstream LLVM reviews, and received a useful clarification on the RISC-V psABI and the ratification lifecycle.
LLVM Weekly #477.

Week of 13th February 2023

After a fair bit of investigation and thinking about reported compatibility issues between GNU and LLVM tools (particularly binutils ld and lld) due to RISC-V extension versioning, posted an RFC outlining the major issues and a proposed fix for what I consider to be a regression in lld.
- Landed a few LLVM patches cleaning up tests related to this. 8b50048 574d0c2, d05e1e9.
- Posted D144353, a proposed fix for the LLD regression due to overzealous checking of extensions/versions when merging RISC-V attributes.
Organised agenda for and ran the bi-weekly RISC-V LLVM contributor call. Key discussion items were the extension versioning related compatibility issue mentioned below and support for emulated TLS (where I'd left some comments the previous week).
Updated my patch (D143172) to register and configure a RISC-V qemu-user based builder with LLVM's staging buildmaster, based on review feedback.
A variety of upstream LLVM reviews. Also landed D143407, marking Zawrs as non-experimental.
LLVM Weekly #476.

Week of 6th February 2023

Left feedback on the proposed RISC-V psABI patch clarifying treatment of empty structs or unions in the FP calling convention. This is a follow-up to the issue I filed on this issue (where I have D142327 queued up for LLVM to fix our incorrect handling).
Responded to a question on LLVM's Discourse about zicsr and zifencei support in LLVM. As noted, the issue is that we haven't moved RV32I/RV64I 2.1 yet which split out Zicsr and Zifencei. Unfortunately this is a backwards-incompatible change so requires some care.
Worked with a colleague trying to reproduce an assertion failure in his committed patch adding support for WebAssembly externref in Clang that appeared only on an MSan buildbot. The sanitizers project guidance is useful for this, but I ended up rolling a slightly hacky script as I stepped through each part of the multi-stage build and test sequence.
Left my thoughts on a proposed RISC-V preprocessor define to specify support and for and performance of misaligned loads/stores. I like the idea of the define, but prefer sticking to the Zicclsm terminology introduced in the RISC-V profiles.
Posted patch D143507 to mark RISC-V Zawrs as non-experimental, after confirming there were no relevant changes between the implemented 1.0-rc3 spec and the ratified version.
A series of WebAssembly GC type related patches remains a work in progress downstream, but I landed a couple of related minor test cleanups. 604c9a0, 3a80dc2.
Many upstream RISC-V LLVM reviews.
LLVM Weekly #475.

Article changelog

2023-04-03: Added notes for the week of 27th March 2023.
2023-03-27: Added notes for the week of 20th March 2023.
2023-03-20: Added notes for the week of 13th March 2023.
2023-03-13: Added notes for the week of 6th March 2023.
2023-03-06: Added notes for the week of 27th February 2023.
2023-02-27: Added in a forgotten note about trivial buildbot doc improvements.
2023-02-27: Initial publication date.

What's new for RISC-V in LLVM 16

2023-03-18T12:00:00Z

LLVM 16.0.0 was just released today, and as I did for LLVM 15, I wanted to highlight some of the RISC-V specific changes and improvements. This is very much a tour of a chosen subset of additions rather than an attempt to be exhaustive. If you're interested in RISC-V, you may also want to check out my recent attempt to enumerate the commercially available RISC-V SoCs and if you want to find out what's going on in LLVM as a whole on a week-by-week basis, then I've got the perfect newsletter for you.

Documentation

LLVM 16 is the first release featuring a user guide for the RISC-V target (16.0.0 version, current HEAD. This fills a long-standing gap in our documentation, whereby it was difficult to tell at a glance the expected level of support for the various RISC-V instruction set extensions (standard, vendor-specific, and experimental extensions of either type) in a given LLVM release. We've tried to keep it concise but informative, and add a brief note to describe any known limitations that end users should know about. Thanks again to Philip Reames for kicking this off, and the reviewers and contributors for ensuring it's kept up to date.

Vectorization

LLVM 16 was a big release for vectorisation. As well as a long-running strand of work making incremental improvements (e.g. better cost modelling) and fixes, scalable vectorization was enabled by default. This allows LLVM's loop vectorizer to use scalable vectors when profitable. Follow-on work enabled support for loop vectorization using fixed length vectors and disabled vectorization of epilogue loops. See the talk optimizing code for scalable vector architectures (slides) by Sander de Smalen for more information about scalable vectorization in LLVM and introduction to the RISC-V vector extension by Roger Ferrer Ibáñez for an overview of the vector extension and some of its codegen challenges.

The RISC-V vector intrinsics supported by Clang have changed (to match e.g. this and this) during the 16.x development process in a backwards incompatible way, as the RISC-V Vector Extension Intrinsics specification evolves towards a v1.0. In retrospect, it would have been better to keep the intrinsics behind an experimental flag when the vector codegen and MC layer (assembler/disassembler) support became stable, and this is something we'll be more careful of for future extensions. The good news is that thanks to Yueh-Ting Chen, headers are available that provide the old-style intrinsics mapped to the new version.

Support for new instruction set extensions

I refer to 'experimental' support many times below. See the documentation on experimental extensions within RISC-V LLVM for guidance on what that means. One point to highlight is that the extensions remain experimental until they are ratified, which is why some extensions on the list below are 'experimental' despite the fact the LLVM support needed is trivial. On to the list of newly added instruction set extensions:

Experimental support for the Zca, Zcf, and Zcd instruction set extensions. These are all 16-bit instructions and are being defined as part of the output of the RISC-V code size reduction working group.
- Zca is just a subset of the standard 'C' compressed instruction set extension but without floating point loads/stores.
- Zcf is also a subset of the standard 'C' compressed instruction set extension, including just the single precision floating point loads and stores (c.flw, c.flwsp, c.fsw, c.fswsp).
- Zcd, as you might have guessed, just includes the double precision floating point loads and stores from the standard 'C' compressed instruction set extension (c.fld, c.fldsp, c.fsd, c.fsdsp).
Experimental assembler/disassembler support for the Zihintntl instruction set extension. This provides a small set of instructions that can be used to hint that the memory accesses of the following instruction exhibits poor temporal locality.
Experimental assembler/disassembler support for the Zawrs instruction set extension, providing a pair of instructions meant for use in a polling loop allowing a core to enter a low-power state and wait on a store to a memory location.
Experimental support for the Ztso extension, which for now just means setting the appropriate ELF header flag. If a core implements Ztso, it implements the Total Store Ordering memory consistency model. Future releases will provide alternate lowerings of atomics operations that take advantage of this.
Code generation support for the Zfhmin extension (load/store, conversion, and GPR/FPR move support for 16-bit floating point values).
Codegen and assembler/disassembler support for the XVentanaCondOps vendor extension, which provides conditional arithmetic and move/select operations.
Codegen and assembler/disassembler support for the XTHeadVdot vendor extension, which implements vector integer four 8-bit multiple and 32-bit add.

LLDB

LLDB has started to become usable for RISC-V in this period due to work by contributor 'Emmer'. As they summarise here, LLDB should be usable for debugging RV64 programs locally but support is lacking for remote debug (e.g. via the gdb server protocol). During the LLVM 16 development window, LLDB gained support for software single stepping on RISC-V, support in EmulateInstructionRISCV for RV{32,64}I, as well as extensions A and M, C, RV32F and RV64F, and D.

Short forward branch optimisation

Another improvement that's fun to look more closely at is support for "short forward branch optimisation" for the SiFive 7 series cores. What does this mean? Well, let's start by looking at the problem it's trying to solve. The base RISC-V ISA doesn't include conditional moves or predicated instructions, which can be a downside if your code features unpredictable short forward branches (with the ensuing cost in terms of branch mispredictions and bloating branch predictor state). The ISA spec includes commentary on this decision (page 23), noting some disadvantages of adding such instructions to the specification and noting microarchitectural techniques exist to convert short forward branches into predicated code internally. In the case of the SiFive 7 series, this is achieved using macro-op fusion where a branch over a single ALU instruction is fused and executed as a single conditional instruction.

In the LLVM 16 cycle, compiler optimisations targeting this microarchitectural feature were enabled for conditional move style sequences (i.e. branch over a register move) as well as for other ALU operations. The job of the compiler here is of course to emit a sequence compatible with the micro-architectural optimisation when possible and profitable. I'm not aware of other RISC-V designs implementing a similar optimisation - although there are developments in terms of instructions to support such operations directly in the ISA which would avoid the need for such microarchitectural tricks. See XVentanaCondOps, XTheadCondMov, the previously proposed but now abandoned Zbt extension (part of the earlier bitmanip spec) and more recently the proposed Zicond (integer conditional operations) standard extension.

Atomics

It's perhaps not surprising that code generation for atomics can be tricky to understand, and the LLVM documentation on atomics codegen and libcalls is actually one of the best references on the topic I've found. A particularly important note in that document is that if a backend supports any inline lock-free atomic operations at a given size, all operations of that size must be supported in a lock-free manner. If targeting a RISC-V CPU without the atomics extension, all atomics operations would usually be lowered to __atomic_* libcalls. But if we know a bit more about the target, it's possible to do better - for instance, a single-core microcontroller could implement an atomic operation in a lock-free manner by disabling interrupts (and conventionally, lock-free implementations of atomics are provided through __sync_* libcalls). This kind of setup is exactly what the +forced-atomics feature enables, where atomic load/store can be lowered to a load/store with appropriate fences (as is supported in the base ISA) while other atomic operations generate a __sync_* libcall.

There's also been a very minor improvement for targets with native atomics support (the 'A' instruction set extension) that I may as well mention while on the topic. As you might know, atomic operations such as compare and swap that are lowered to an instruction sequence involving lr.{w,d} (load reserved) and sc.{w,d} (store conditional). There are very specific rules about these instruction sequences that must be met to align with the architectural forward progress guarantee (section 8.3, page 51), which is why we expand to a fixed instruction sequence at a very late stage in compilation (see original RFC). This means the sequence of instructions implementing the atomic operation are opaque to LLVM's optimisation passes and are treated as a single unit. The obvious disadvantage of avoiding LLVM's optimisations is that sometimes there are optimisations that would be helpful and wouldn't break that forward-progress guarantee. One that came up in real-world code was the lack of branch folding, which would have simplified a branch in the expanded cmpxchg sequence that just targets another branch with the same condition (by just folding in the eventual target). With some relatively simple logic, this suboptimal codegen is resolved.

; Before                 => After
.loop:                   => .loop
  lr.w.aqrl a3, (a0)     => lr.w.aqrl a3, (a0)
  bne a3, a1, .afterloop => bne a3, a1, .loop
  sc.w.aqrl a4, a2, (a0) => sc.w.aqrl a4, a2, (a0)
  bnez a4, .loop         => bnez a4, .loop
.aferloop:               =>
  bne a3, a1, .loop      =>
  ret                    => ret

Assorted optimisations

As you can imagine, there's been a lot of incremental minor improvements over the past ~6 months. I unfortunately only have space (and patience) to highight a few of them.

A new pre-regalloc pseudo instruction expansion pass was added in order to allow optimising the global address access instruction sequences such as those found in the medany code model (and was later broadened further). This results in improvements such as the following (note: this transformation was already supported for the medlow code model):

; Before                            => After
.Lpcrel_hi1:                        => .Lpcrel_hi1
auipc a0, %pcrel_hi1(ga)            => auipc a0, %pcrel_hi1(ga+4)
addi a0, a0, %pcrel_lo(.Lpcrel_hi1) =>
lw a0, 4(a0)                        => lw a0, %pcrel_lo(.Lpcrel_hi1)(a0)

A missing target hook (isUsedByReturnOnly) had been preventing tail calling libcalls in some cases. This was fixed, and later support was added for generating an inlined sequence of instructions for some of the floating point libcalls.

The RISC-V compressed instruction set extension defines a number of 16-bit encodings that map to a 32-bit longer form (with restrictions on addressable registers in the compressed form of course). The conversion 32-bit instructions 16-bit forms when possible happens at a very late stage, after instruction selection. But of course over time, we've introduced more tuning to influence codegen decisions in cases where a choice can be made to produce an instruction that can be compressed, rather than one that can't. A recent addition to this was the RISCVStripWSuffix pass, which for RV64 targets will convert addw and slliw to add or slli respectively when it can be determined that all the users of its result only use the lower 32 bits. This is a minor code size saving, as slliw has no matching compressed instruction and c.addw can address a more restricted set of registers than c.add.

Other

At the risk of repeating myself, this has been a selective tour of some additions I thought it would be fun to write about. Apologies if I've missed your favourite new feature or improvement - the LLVM release notes will include some things I haven't had space for here. Thanks again for everyone who has been contributing to make the RISC-V in LLVM even better.

If you have a RISC-V project you think me and my colleagues and at Igalia may be able to help with, then do get in touch regarding our services.

Article changelog

2023-03-19: Clarified that Zawrs and Zihintntl support just involves the MC layer (assembler/disassembler).
2023-03-18: Initial publication date.

Muxup

Clarifying instruction semantics with P-Code

instruction_to_pcode

Other approaches

Reflections on ten years of LLVM Weekly

Motivation and purpose

Readership and content

How it works

How you can help

Miscellaneous thoughts

Looking forwards and thanks

Let the (terminal) bells ring out

BEL

Terminal emulator support

Ownership you can count on

Introduction

Ownership you can count on and the Gel language

Influence and adoption of the paper's ideas

Conclusion and other related work

Storing data in pointers

Introduction

Storage in upper bits assuming 48-bit virtual addresses

What if virtual addresses are wider than 48 bits?

Top byte ignore and similar features

Storing data in least significant bits

Some real-world examples

Fin

What's new for RISC-V in LLVM 17

Code size reduction extensions

Vectorization

Other ISA extensions

Other additions and improvements

2023Q2 week log

Week of 29th May 2023

Week of 22nd May 2023

Week of 15th May 2023

Week of 17th April 2023

Week of 10th April 2023

Week of 3rd April 2023

Updating Wren's benchmarks

New results

Old results

Observations

Appendix: Benchmark script

2023Q1 week log

Week of 27th March 2023

Week of 20th March 2023

Week of 13th March 2023

Week of 6th March 2023

Week of 27th February 2023

Week of 20th February 2023

Week of 13th February 2023

Week of 6th February 2023

What's new for RISC-V in LLVM 16

Documentation

Vectorization

Support for new instruction set extensions

LLDB

Short forward branch optimisation

Atomics

Assorted optimisations

Other