[m-rev.] for review: Overhaul of the Syntax chapter of the reference manual

[Mark Brown]

Hi Julien.

On Thu, Sep 1, 2022 at 12:36 AM Julien Fischer <jfischer at opturion.com> wrote:
>
>
> Hi Mark,
>
> On Wed, 31 Aug 2022, Mark Brown wrote:
>
> > This makes some significant changes to the Syntax chapter. The
> > motivation for these changes comes largely from what questions were
> > needing to be asked by programmers with a non-Prolog background who
> > were learning Mercury, and what things they were confused about.
> >
> > The texinfo is for review by Peter, but I've also attached a tarball
> > of the split html for anyone else to look through (and pdf and info,
> > in case anyone cares).
>
> Here are some initial comments regarding your proposed changes to the
> structure of the reference manual.
>
> > commit 923a62ae30a236ba11b61020c3636e6909e67925
> > Author: Mark Brown <mark at mercurylang.org>
> > Date:   Tue Aug 30 07:53:25 2022 +1000
> >
> >     Overhaul of the Syntax chapter of the reference manual
> >
> >     There are three main aims for this change:
> >
> >     1. Don't rely so much on users having knowledge of Prolog (ideally we
> >     wouldn't require it at all, but this change probably doesn't fully achieve
> >     that).
>
> Ideally, the reference manual would contain one reference to Prolog. That
> reference would point interested readers to the Prolog transition guide.

Agreed - that would make it very clear that it isn't required.

We still refer to things like SLDNF resolution, though, which in
practice means needing to know about Prolog. So I can't say that I was
aiming to remove dependencies entirely.

>
> > Present the information more linearly, i.e., without requiring so many
> > forward references.
> >
> >     2. Make important information less likely to be overlooked. As it stands,
> >     we give a lot of semantics in the Syntax chapter where users wouldn't
> >     expect to find it (they're more likely to just skim over such a chapter).
>
> I agree with this; I don't agree with how you've decided to address it (see
> below).
>
> >     We also currently break the chapter up into small sections, some of which
> >     have little information. This works well in the PDF version, but in split
> >     html and info versions where navigation is required it is a pain.
>
> One reason for having those small sections is that they appear in the
> table of contents. In the existing version, it's _really_ obvious where
> to look, to give an example, for the list of characters that are
> considered whitespace.
>
> Building the HTML version of the reference manual with --split=chapter
> would appear to address your concerns for the HTML version.  (The Info
> format and the tools for viewing it that I am familiar seem to have a
> rather fixed view about how documents are presented.)
>
> > People tend to forget information each time they navigate, and are likely to
> > find themselves going back and forth between nodes.
>
> Which suggests that splitting at nodes (which is the texinfo default) is
> not the right thing to do for this document.

Agreed. And I didn't know about splitting by chapter - I will check it out.

For the whitespace and other early subsections, I wasn't aiming for
"really obvious" (Lexical syntax/structure is sufficiently obvious,
imho) but more "not dry and dull". I think it is keeping with the
style of the rest of the manual to present the information like an
article, rather than in an overly structured way. I also think the way
the table splits up the text is more aesthetically pleasing, but I
won't argue that point any further ;-)

>
> ...
>
> >     Changes to document structure
> >     -----------------------------
> >
> >     Sections and subsections of the Syntax chapter have been merged, to provide
> >     a more reasonable amount of information at each node.
>
>
> >     The Syntax chapter
> >     as a whole has been split into two chapters, Syntax and Semantics (the
> >     existing Semantics chapter is renamed Formal Semantics). The new structure
> >     is as follows:
>
> I think "Semantics" is a problematic name: there's a load of syntax
> stuff in there as well. Rather than "Semantics" I suggest simply calling
> it "Goals and Expressions".

Well, there's some pretty significant things other than goals and
expressions, and most of the remaining chapters provide syntax along
with the semantics. If you want a more accurate name then it could be
"Clause semantics" for example. If you just want to suggest that it's
not _all_ of the language semantics it could be called "Basic
semantics" or something like that.

>
> Thinking about this a bit more: there's no particular reason that we need to
> have a chapter named "Syntax" that introduces all the syntax up front. Here,
> for the sake of discussion, is an alternative structure for the beginning of
> the reference manual:
>
>       1. Introduction
>       2. Lexical structure
>          2.1 Character set
>          2.2 Whitespace
>          2.3 Comments
>          2.4 Tokens
>          2.5 Operators

I think the difference here mainly depends on whether we want to use
the "article" style or the "structured style". The structure you've
given here will definitely be familiar to just about any programmer,
so I'll ask a few non-Prolog programmers who have learned Mercury what
they think (aside from waiting for others on this list). I'll be happy
to change to this structure if need be.

>       3. Terms

This is better placed under something entitled "Syntax", in my view,
otherwise it just sounds like a glossary.

One of the reasons why I wanted to make a clear distinction between
syntax and semantics, at least early on, is so I could explain why you
can use operators all over the place (including where you don't want
to, like character literals), but you can't e.g. add two integers
together anywhere you please because you need to import a definition.
Also, so I could explain why you sometimes need parentheses in weird
places, like declarations.

>       4. Goals and Expressions
>          4.1 Goals
>          4.2 DCG-goals
>          4.3 Expressions
>          4.4 Variable scoping
>          4.5 Implicit quantification
>          4.6 Elimination of double negation
>       5. State variables
>       6. Items

>       7. Types
>       ...
>
> >     Syntax
> >
> >     - The opening blurb covers the material from "Syntax overview" and
> >     "Character set". The definitions are put first, and comparisons with
> >     Prolog are put after (in general we try to make such comparisons only
> >     as secondary information).
> >
> >     - "Lexical syntax" covers the material from "Whitespace" and "Tokens".
> >     We also provide comment syntax. Line number directives are no longer
> >     considered tokens, since we are supposed to ignore them when parsing
> >     terms anyway.
>
> I would title this subsection "Lexical structure".  That is a fairly common name
> for it in other language specifications (e.g. Java, C#, Haskell, Swift etc).

Sure.

>
> I would make the existing sections like "Whitespace" and "Tokens" into
> titled subsubsections of this.
>
> >
> >     - "Terms" covers the material previously in that section. It adds a summary
> >     in the form of grammar rules (but makes it clear all the information is
> >     also in the prose), and a term normalization procedure that is _not_
> >     written in pseudo-Mercury. The grammar is there for people who are likely
> >     to skip the prose but less likely to skip a grammar, and the prose is there
> >     because that's the best way to describe things.
> >
> >     - "Builtin operators" covers the same material as before.
> >
> >     - "Items" covers the material previously in "Items", "Declarations",
> >     "Facts", "Rules", and "DCG-rules". It also adds a summary in the form of
> >     grammar rules.
> >
> >     Semantics
> >
> >     - I've written an introductory section on declarative programming, which
> >     informally introduces the declarative and operational semantics (and why
> >     you want them both).
>
> The position of this seems fairly arbitrary; given its tone and content I feel
> it would be better placed in an extended version of the introduction to the
> entire document or in a separate chapter just after the introduction.

Yes, I guess you could say it is arbitrary. The question is whether it
accurately characterizes our claims to be a declarative programming
language.

We use the terms "declarative semantics" and "operational semantics"
throughout the remainder of the manual so we need to explain what
we're talking about, and the formal semantics is pretty steep for
anyone who doesn't already know Prolog. Would anyone else know what
the completion of the clauses is? One thing I can say for certain is
that there are very good programmers out there who don't know what
unification is.

And it doesn't make sense in an introduction, since it uses examples
of Mercury syntax which it asks you to already understand.

Regarding one claim I've made:
> One notable characteristic of declarative languages, however, is that they have both a declarative and an operational semantics.

I'd agree that this is going too far; I really just want to say that
it is notable about Mercury.

Thanks for the initial feedback.

Cheers,
Mark

>
> Cheers,
> Julien.