jlm-blog
~jlm

15-Apr-2020

One line of javascript to be console friendly

Filed under: web — jlm @ 08:38

From early on in styling this blog, I’ve had the sidebar on the top right, with the main blog content flowing against it, then after it ends lower in the page, below it. (You won’t notice this if your browser is very wide, as I cap the width of the main body at 50 em, but if you shrink it some you’ll see this.) Frankly, I’m surprised this is so unusual in blog and related webpage themes, which generally keep the space beneath sidebars empty, just wasting that space. (At least, when it’s not filled with ads.) This is super easy to do: in the page’s layout, have the sidebar’s markup come before the main body and give it the CSS style rule float: right. The browser will do everything from there, filling the page with the content that follows the sidebar’s markup, line wrapping when it hits the sidebar while also filling the space below it, doing a better job than most javascript-based layout renderings do.

There’s only one thing I consider a problem with this technique. In text console browsers (lynx, links, elinks, etc.), it shows the sidebar before the main content because the sidebar is in the markup earlier, which is way less friendly than if it appeared after. If I move it after the body, then it shows up better on the console, but now it’s at the bottom of the page in the graphical browsers used by all sane people, which is no good. I’ve had a soft spot for the console browsers from almost my first encounter with the web, long before I started this blog. The reasons for that is another story, but there’s no way I’m going to sacrifice the view in graphical browsers for improving the console browser experience, so I’ve kept the sidebar earlier in the markup than the main content all 15 years I’ve run this blog. And all that time I’ve felt tiny, insignificant tinges of regret for not offering a better experience for the extremely rare visitor from a console browser.

Every so often I’d think about tweaking WordPress to swap the order of the main content and the sidebar if the user-agent was a console browser, but that seemed inelegant and ugly, plus likely to be a pain with doing it in PHP, learning the appropriate WordPress internals, and likely having to carry a patch forward across WordPress versions. Or I’d think about doing it in javascript: have the sidebar be after the content in the markup so it shows up well in the console browsers that don’t execute javascript, but run a script to move it earlier in the graphical browsers that do. But then I’d have to learn javascript and DOM manipulation, which seemed like it was going to be a pain too.

Well, it turns out it is so not a pain it’s almost funny. I’m familiar with giving elements “names” with the id HTML attribute, for use in sub-page links and CSS #-references. Those names turn out to be great for getting DOM handles in javascript: just call document.getElementById("TheName"). And moving an element from where it is in the original markup to inside another element is just NewEnclosingElement.appendChild(ElementBeingMoved). Putting those together, I simply:

  • Replaced the <?php get_sidebar(); ?> in the theme’s header.php file with <div id="sidebar"></div>
  • Gave that element a float: right CSS rule
  • Added this to the theme’s footer.php file:
    <?php get_sidebar(); ?>
    <script type="text/javascript">
    document.getElementById("sidebar").appendChild(document.getElementById("menu"))
    </script>
    

(WordPress names the <div> that encloses the sidebar “menu”. I picked the name “sidebar” for the <div> which “menu” gets moved into.)

And … that’s it. The blog now looks like I want it to in both lynx and firefox. I probably shouldn’t have waited 15 years to look into this!

31-Mar-2020

TIL: gshadow has user lists

Filed under: linux — jlm @ 21:37

While I was chatting (well, rapid-fire emailing) with a friend who works as a system administrator, he dropped a bit of Linux trivia on me: It’s not just /etc/group which has user lists, the /etc/gshadow file also has user lists — more than /etc/group does, even! After the crypted password is a list of group administrators, then a list of shadow members. The former have the ability to change the group’s password as well as its membership using the gpasswd command. The latter can make the group be their primary group by calling newgrp without needing a password. (See man gshadow.)

21-Mar-2020

Simple Scala puzzle

Filed under: programming — jlm @ 10:46

Who can guess what this tiny* Scala program does?

object O extends App {
    val x = 123456789L
    val y = 0F
    val z = x - (if (true) x else y)
    println(z)
}

(Apparently this mysterious snippet had been going around a while ago, but I only recently encountered a variation.)

Answer: Prints “−3.0”.

Why?

Short answer: The if/else causes x to get promoted to Float, which rounds it up to 123456792.

Long answer after the fold.

(more…)

17-Feb-2020

Unit tests are not enough

Filed under: programming — jlm @ 20:48

[I wrote this for another blog back in 2011, but it was lost to public view when that blog disappeared. It’s still valid today, though!]

In one of our datastructures, there’s an entry for a 64-bit unique resource identifier. This turned out to be a little too vague a description: Is it an 8-byte array, a 64-bit wide integer, or an instance of a 64-bit UUID class? One of our developers though one thing, and another thought another. No problem, right? The first compile by the one who chose wrong will show the type error. Except we’re using Python, so no type checking until the code runs. Still no problem, because our devs wrote unit tests, so the type error will show up on their first run, right?

Unfortunately, unit tests help not an iota in situations like this. The developer expecting UUID objects writes unit tests which consume UUID objects, and the dev expecting byte arrays writes unit tests which produce byte arrays.

So, integration tests, then? It turns out our devs did these too, verifying that their code integrated with the fancy datastructure class, but this still didn’t catch the mismatch! This is because the datastructure class doesn’t actually do anything with the resource identifier that would trigger a type check: it just accepts it as an entry in a slot when an item is inserted, and gives it out as one of the slot entries when items are looked up, and Python is happy to let an instance of any type pass through unchecked.

It’s only when we plug everything together that this trivial type mismatch shows up. So the moral: Make end-to-end tests early. You need them to turn up even problems as basic as this, and the earlier you find problems, the better.

2-Feb-2019

Less is more, CPU edition

Filed under: general, programming — jlm @ 17:11

I fixed an interesting bug recently. By way of background, I work on industrial robotics control software, which is a mix of time-critical software which has to do its job by a tight deadline (variously between 1 and 10 ms) or a safety watchdog will trigger, and other software that does tasks that aren’t time sensitive. We keep the timing-insensitive software from interfering with the real-time software’s performance by having the scheduler strictly prioritize the deadline-critical tasks over the “best effort” tasks, and further by massively overprovisioning the system with far more CPU and memory than the tasks could ever require.

The bug (as you may have guessed) is that the deadline was sometimes getting missed, triggering the watchdog. What makes the bug interesting is that it was caused by us giving the system too much excess CPU and memory!

The deadline overruns happen when the computer runs best-effort tasks. It only runs those tasks a small fraction of the time, but no matter, they shouldn’t be interfering with the deadline completion at all (and having our software fail its deadlines is unacceptable, of course). The real-time tasks’ peak usage is only about a quarter of the computer’s CPU capacity, and the system gives them that always: 100% of the time they want CPU, they get CPU. They are never delayed, and never have to wait on a resource held by a best-effort task. When they miss the deadlines, it’s always because they’ve gotten jobs that take the usual amount of work to complete, and had their usual amount of time to do it, yet the jobs somehow just take more time to finish. There’s no resource contention, nor are the caches an issue.

Yet when the best-effort tasks execute, the calculations done by the real-time tasks run slower on the same CPU for the same job with the same cache hit rate and with no memory or I/O contention. What’s going on? After hitting some false leads, I discovered that they’re going slower because the CPUs are running at a lower frequency. It turns out the CPUs’ clocks have been stepped down because they’re getting too hot. (Yes, the surrounding air is much warmer than it “should” be, but we don’t have a choice about that.)

The computer is hot because all the CPUs are going at full blast, because all of the best-effort tasks are executing because the computer can fit them all in memory. Half a minute later, the tasks are done, the computer cools off, the CPU frequency gets stepped back up, and the deadline overruns cease. An hour later, the hourly background tasks all go again, the fans spin up to full, but they’re not enough and the CPU frequency steps down, hence the watchdog alerts about missed deadlines.

Okay, maybe we should change the background tasks so they execute in a staggered fashion. But before thinking about how I might do that, I tried disabling ⅔ of the computer’s CPUs instead. The hourly processing now takes 200s instead of 30, but it’s still far below 3600, and that’s what matters. Now the computer stays nice and cool, so the CPUs stay nice and quick, and the deadlines all get met. We were using too many CPUs to get the calculations we needed to get done fast, done fast. Who’d’a thunk?

20-Oct-2018

Scheme got incrementally better

Filed under: programming — jlm @ 08:34

Almost a decade ago, I wrote about how the venerable and elegant but third-tier programming language Scheme had no way to access the program arguments that would work across different implementations, and pointed out that type of large stumbling block tends to hinder the adoption of a language. The last time I worked on a serious project in Scheme was only a little later. So not much news from the Schemeoverse has reached me since then. Which is why I just learned that four years after that post, five years previous to the present day, the Scheme community decided to fix that and standardized on option (c) from that post: (command-line) gets you your program’s arguments. Progress marches on! In the case of Scheme, that march is just veerrryyyy slow.

25-Aug-2018

Open source CPUs are not new

Filed under: general — jlm @ 23:25

There’s this video going around “An Open Source CPU!?” about SiFive’s implementation of RISC-V being “an open source CPU”, but it’s so duplicitous that I wanted to tear out my hair as I watched it. RISC-V is an interesting and worthy project, but the video is misleading in so many ways it’s hard to know where to begin.

For one thing, the x86-16 ISA wasn’t just hacked up in a few weeks from nothing. Intel’s architects spent months on it, and it was based on Intel’s 8-bit 8080 ISA, which was in turn informed by the experience of their earlier 8008 ISA. It had also already been on the market for a few years by the time IBM got around to the PC. For another thing, the x86-32 ISA was hardly lacking competition during its era of desktop dominance, with Motorola’s 68000, IBM’s Power Architecture, and DEC’s Alpha all big-name-backed ISAs trying to displace it. When the time came for 64-bit PCs, even Intel itself failed in its attempt to push a new kind of ISA for the PC market with its Itanium architecture.

Regarding the RISC/CISC divide, it is true that CISC processors don’t execute their ISA instructions directly in hardware now, but instead run RISC microcode (because RISC instruction-handling circuitry executes faster and is easier to design) for a CISC ISA interpreter. But that doesn’t mean RISC is best for application or system code, because microcode has no instruction cache! In the embedded systems and mobile-device markets, ARM had to abandon the “elegant” simplicity of their RISC ISA and supplement it with non-RISC “Thumb” and then “Thumb-2” extensions to raise its code density (important on small processors with small instruction caches) to keep x86 ISA microcontrollers, with their dense CISC code and ever-decreasing prices, at bay. As is often the case when competing technologies butt heads, the best solution is to find a way to use the advantages of both!

The video’s title is “An Open Source CPU!?” — and this is its biggest lie, because SiFive’s CPU being open is presented as something new, which is very much not true. Even if you dismiss all CISC designs as unworthy, there are open RISC specs with open source implementations that predate RISC-V, both grassroots and of commercial origin. Furthermore, SiFive’s CPU not only isn’t the first open source RISC processor, it’s not even the first open source RISC-V processor! You can get open source implementations of RISC-V here (VexRiscv) or here (ORCA) or here (Sodor).

This video is just an advertisement of SiFive’s pretending to be an independent informational video, spreading misinformation about the history and present state of microprocessors. (For your amusement, go and pause the video around 6m12s — the presenter is stating the inferiority of other open ISAs compared to RISC-V while showing a changelog on GitHub. But the changelog shown is not that of anything to do with another ISA, but the opposite: it’s actually the changelog for the port of gdb to RISC-V! The presenter could have grabbed a screenshot from the GitHub pages of the ZPU project with insignificant additional effort and shown something that was actually applicable to his claim, but once you’ve stepped off the straight-and-narrow path of honesty onto the wide highway of dishonesty, why bother?)

In short, that video is complete and utter BS, the presenter should be downvoted into YouTube oblivion, you should never pay one red cent for anything from SiFive, and RISC-V deserves better. It’s strong enough that it should be able to fight honestly, and having its proponents descend to dishonest shilling for one corporation is disgraceful.

5-Oct-2017

Tutorial for building .so files

Filed under: linux, programming — jlm @ 18:34

Exactly how one goes about building .so files* isn’t widely understood, and the documentation on it is overcomplicated IMHO. So, I figure the web needs a tutorial demonstrating how to build an extremely simple .so file. Et voilà, here’s a step by step guide for building a .so file on Linux or a similarly flavored Unix using a GNU-compatible toolchain.

We’re going to make two .o files, “call_me.o” and “die.o”, and combine them into a single “call_me_and_die” shared library. Making a library is very much like making an executable, just with some tweaks to the build commands and some extra steps at the end. To make our “call_me_and_die” library, we start the way we would with a “call_me_and_die” executable: writing a .h file for the common function signatures. File so_tut.h:

extern void call_me(void);
extern void die_die_die(void);

(more…)

11-Sep-2017

EBUSY: Another decade, and still sucking

Filed under: general, linux, programming — jlm @ 12:40

Today’s xkcd riffs a common frustration about busy files: programs that can’t do their intended operations on them rarely (if ever) specify the file that’s busy or tell the user which programs are using the file — and without that knowledge, the user can’t fix the problem.

There are tools that can help you discover those things, as I describe using ten years ago, but I’d already been running into EBUSY problems for ten years by then, and the old guard before me for another ten years beyond that. Why haven’t things improved?

Sometimes file_operation(filename) fails — hey, this happens, busy files are a thing, ain’t nothing your program can do about that. But report to the user that file_operation on filename failed for reason strerror(errno)! Not reporting the filename is just sloppy programming.

It’s not like the tools have gotten any better. What, you think it’s acceptable to make the user trace your program, looking through a haystack of every system call to find the needle which is the program’s critical failure? Is that work you’re putting on the user’s shoulders less than sticking the filename in the error message? This has sucked for two generations now. Please help making it suck less.

4-Mar-2017

twitcode #4: reverse diff — using the right tool for the job

Filed under: programming — jlm @ 20:29

Mercurial’s  hg diff  command supports a  --reverse  option which shows the regular diff output except it reverses the sense of the comparison — i.e., it goes from the “destination” to the “source” (git diff  supports this action too, but as the flag “-R”). Most of the time you want the ordinary “forward” sense, but occasionally the reverse sense comes in handy, and that’s why that option’s there. On rare occasion, I’ll even want to do this to files not under version control, but the regular system diff doesn’t support this feature.

So, after hitting this deficiency again recently, I decided to write up my own reverse-diff command which would swap its last two arguments and call diff. I started with the shell, as dealing with command arguments and calling programs is its forte. But it turned out to be surprisingly difficult to do stuff like copy the argument list or mess around with the end and near-end of the argument list, which I thought would be dead-simple operations. After futzing around with shell variables and parameters and the various options for variable/parameter expansion for something like 25 minutes, I came to my senses and did it in something like two minutes using C, where nothing’s going to interpret any kind of data as anything unless you explicitly request it to, and array manipulation is built in with clean syntax. All I had to do was swap argv[argc-1] and argv[argc-2] then execvp("diff", argv), easy peasy.

And if I golf argc and argv into c and v, then it fits in 130 chars [source]:
#include <unistd.h>
int main(int c, char**v) {
 if (c>2) { char*t=v[c-2]; v[c-2]=v[c-1]; v[c-1]=t; }
 return execvp("diff", v);
}

I could probably omit the check for ≥2 arguments, as the system diff doesn’t support the convention that a missing file argument means to use stdin (instead, it treats a filename of - (single hyphen) as representing stdin), but perhaps it’ll be used by somebody who’s installed an enhanced diff program.

I’m also amused that each of my “twitcodes” has been in a different language: shell, perl, python, and now C.

Powered by WordPress