Missing features that Crater found

[notriddle]

We’ve been looking into using this project to implement built-in math support in Rustdoc.

To test if this library is a suitable replacement for what people are currently using (JS libraries, mostly), I ran a Rust ecosystem test build with pulldown-cmark’s math support and this crate. I set up to fail noisily if
this library failed to parse, so that the build failures gave me a list of issues that the user would encounter if we shipped this library to them as-is.

https://gist.github.com/notriddle/108fe255ffa9f490ed8ade262935eb53

rust-lang/rust#151918

The results show many cases where $ is triggering a math span where the author didn’t intend one, and a bunch of problems caused by markdown escaping weirdness. Those problems are not your problems.

Here’s the problems that seem to exist in this library, and affect more than one crate:

• \textrm
• \textsf
• \ket
• \floor
• \Z
• \rm
• \R
• \mathbin
• \space
• \small
• \newline
• \bold

[tmke8]

Ah, that's a useful list. The hardest from this list will be \mathbin (there is a tracking issue for it here: #523) . I can implement the other ones.

By the way, I recently added a flag to pass through unknown commands to the output:

math-core/crates/math-core/src/lib.rs

Lines 118 to 120 in 7099890

	/// If `true`, unknown commands will be rendered as red text in the output, instead of
	/// returning an error.
	pub ignore_unknown_commands: bool,

That might be an option.

[tmke8]

In #829 I have added:

• \textrm and \textsf: these require additional CSS as detailed here in the README: https://github.com/tmke8/math-core#css-for-text-mode-fonts
• \ket (I also added \bra while I was at it)
• \Z and \R
• \space (alias for \ )
• \newline (I copied KaTeX's behavior for it even though that's a bit sketchy)
• \bold (alias for \mathbf)

EDIT: I added a few that only appear once in the crater output in #830 and #831.

I haven't yet added the following:

\floor

Neither KaTeX nor MathJax supports this, so I'm inclined to not support it either. But it's very easy to add as a custom macro:

let macros = vec![
    ("floor".to_owned(), "{\\lfloor #1\\rfloor}".to_owned()),
];
let config = MathCoreConfig {
    macros,
    ..Default::default()
};

let converter = LatexToMathML::new(config).unwrap();

\rm

This is the inline version of \mathrm: {\rm hello}. We can definitely support this, but it's slightly more involved. Maybe I can do it over the weekend.

\mathbin

As I said above, this is the trickiest. I have some ideas for how to do it, but it might take a while.

\small

I'm not sure about this. I looked at where this is used in your crater run and saw it was these two:

• https://github.com/kjfootman/jm_math/blob/37bebeb9ec2ac89fd71e4b43cba6a06df4132a40/src/linear_algebra/matrix.rs#L22C7-L24
  • I can't really tell what the purpose of \small is here
• and many hits in this project: https://github.com/johnstonskj/rust-asdi, for example here: https://github.com/johnstonskj/rust-asdi/blob/7dec0ce8c394cabb96204c723d83dc1d717249c5/src/features.rs
  • my guess is that the author thought the math font was too big compared to their text font; \small is used in almost every formula that is embedded in the text

Both use \small in math mode, where it is actually not allowed in LaTeX. In math mode, you're supposed to use \scriptstyle or \scriptscriptstyle. If you want to use \small, you can only use it in \text{}. It seems KaTeX and MathJax support \small also in math mode and I can kind of see why --- \scriptstyle is for most purposes too small --- but I'm not sure I want to copy that behavior. I could maybe add a lenient mode to the converter.

[tmke8]

I looked at a few of the

unknown command [

errors. As far as I can tell, they're due to escaping within markdown tables (not sure why [ needs to be escaped there, though).

[notriddle]

LaTeX Math isn’t really supported in rustdoc right now. The people who’re doing it are rolling it themselves by injecting katex.js into the HTML, and this means their math code is being parsed as inline markdown text and katex is reading the math code from the HTML.

In rustdoc, you wrap something in square brackets to write an intra-doc link, and you write a literal square bracket by backslash escaping it¹. When katex.js does its own parsing, backslash-escaped punctuation has been replaced by the lone punctuation mark. Technically, you don’t need to escape your square brackets if the wrapped text isn’t a valid intra-doc link, but you get a warning.

We added “proper” support for latex math spans to pulldown-cmark, and it fixes these quirks, but the existing docs rely on them.

Footnotes

1. you always knew this, since you’ve used intra-doc links in your own code, but I’m writing this down for the benefit of anyone reading, and to set up context ↩

[tmke8]

\rm is now also supported with #841.

Thanks for the explanation. The fact that [`Vec<T>`] is used for intra-doc links was not on my mind as I was thinking of pure markdown.

[tmke8]

I've now also added preliminary support for \mathbin. The current limitations are that it can only handle a single symbol and that it can only handle symbols that are considered operators by the MathML Core spec (this excludes, e.g., letters). But this should cover most use cases.

For example, this works now:

x \mathbin| y

[tmke8]

By the way, @notriddle , the main reason I'm aware of to maybe not use math-core for rendering math on Rustdoc is that some accents are displayed incorrectly on Safari:

(you can test it in your own browser at https://tmke8.github.io/math-core/test.html )

Temml has implemented a work-around for this, but the work-around makes all accents on all browsers look a little worse, so I'd rather avoid that. I have been trying to find a better work-around but haven't found one so far. I also opened an issue on webkit about it: https://bugs.webkit.org/show_bug.cgi?id=308384

[tmke8]

Update on the Safari issue: I think I have found a solution, which is rather obvious in retrospect: just lower the glyph in the font file. In my initial tests this worked well. Changing all the affected glyphs in all the fonts and testing it more will take some time though.