[ELPA] New package: xr

classic Classic list List threaded Threaded
45 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[ELPA] New package: xr

Mattias Engdegård-2
This is a proposal to add xr to ELPA.

The xr package is the inverse of rx: it transforms an Emacs regexp string into rx form. Its purpose is to assist in migration to rx-style regexps, something I think a lot more people should do, and help understanding difficult regexps.

It is written to take all features and oddities in the Emacs regexp notation into account. This includes undocumented parts, since such regexps are found in actual code.

The package consists of a single file, right now at https://github.com/mattiase/xr . It is something I have had lying around for some time and found constantly useful, and have now had the opportunity to give some polish.


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Clément Pit-Claudel
On 01/02/2019 09.09, Mattias Engdegård wrote:
> This is a proposal to add xr to ELPA.
>
> The xr package is the inverse of rx: it transforms an Emacs regexp string into rx form. Its purpose is to assist in migration to rx-style regexps, something I think a lot more people should do, and help understanding difficult regexps.
>
> It is written to take all features and oddities in the Emacs regexp notation into account. This includes undocumented parts, since such regexps are found in actual code.
>
> The package consists of a single file, right now at https://github.com/mattiase/xr . It is something I have had lying around for some time and found constantly useful, and have now had the opportunity to give some polish.

Looks great. Here's a feature request: can you combine this with a font-lock rule to display existing regexps in rx syntax?  Something like prettify-symbols-mode, but for regular expressions.

Clément.

Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Eli Zaretskii
> From: Clément Pit-Claudel <[hidden email]>
> Date: Fri, 1 Feb 2019 10:25:12 -0500
>
> Looks great. Here's a feature request: can you combine this with a font-lock rule to display existing regexps in rx syntax?  Something like prettify-symbols-mode, but for regular expressions.

Please, NOT like prettify-symbols-mode!  An overlay string or a
display property that shows the rx syntax should be much better.

TIA

Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Mattias Engdegård-2
In reply to this post by Clément Pit-Claudel
1 feb. 2019 kl. 16.25 skrev Clément Pit-Claudel <[hidden email]>:
>
> Looks great. Here's a feature request: can you combine this with a font-lock rule to display existing regexps in rx syntax?  Something like prettify-symbols-mode, but for regular expressions.

You mean a mode so that when the user is looking at the elisp source

  (if (looking-at "regexp-string") ...

he would actually see

  (if (looking-at (rx translated-expression)) ...

? Could be done, I suppose, but since there is no special syntax for regexp strings, it would probably either need to rely on a list of well-known functions with regexp arguments, or second-guess it from their argument names (like REGEXP or PATTERN). It sounds a bit fragile.

I suggest that someone who would like to explore this in detail build a prototype using xr as the translation engine; its public interface should be quite sufficient for the task.


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Clément Pit-Claudel
In reply to this post by Eli Zaretskii
On 01/02/2019 10.42, Eli Zaretskii wrote:
>> From: Clément Pit-Claudel <[hidden email]>
>> Date: Fri, 1 Feb 2019 10:25:12 -0500
>>
>> Looks great. Here's a feature request: can you combine this with a font-lock rule to display existing regexps in rx syntax?  Something like prettify-symbols-mode, but for regular expressions.
>
> Please, NOT like prettify-symbols-mode!  An overlay string or a
> display property that shows the rx syntax should be much better.
>
> TIA

:) I meant looking like prettify-symbols-mode; I wasn't suggesting to use composition for the implementation.


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Clément Pit-Claudel
In reply to this post by Mattias Engdegård-2
On 01/02/2019 10.51, Mattias Engdegård wrote:

> 1 feb. 2019 kl. 16.25 skrev Clément Pit-Claudel <[hidden email]>:
>>
>> Looks great. Here's a feature request: can you combine this with a font-lock rule to display existing regexps in rx syntax?  Something like prettify-symbols-mode, but for regular expressions.
>
> You mean a mode so that when the user is looking at the elisp source
>
>   (if (looking-at "regexp-string") ...
>
> he would actually see
>
>   (if (looking-at (rx translated-expression)) ...
>
> ? Could be done, I suppose, but since there is no special syntax for regexp strings, it would probably either need to rely on a list of well-known functions with regexp arguments, or second-guess it from their argument names (like REGEXP or PATTERN). It sounds a bit fragile.

I would suggest neither; instead, it would convert only strings that contain regexp constructs (these are the ones that are hard to read in the first place).
That;'s more or less what I do in https://github.com/cpitclaudel/easy-escape, and it works OK.

Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Mattias Engdegård-2
1 feb. 2019 kl. 19.41 skrev Clément Pit-Claudel <[hidden email]>:
>
> I would suggest neither; instead, it would convert only strings that contain regexp constructs (these are the ones that are hard to read in the first place).
> That;'s more or less what I do in https://github.com/cpitclaudel/easy-escape, and it works OK.

I see. In that case, it seems that you are ideally positioned to make such an attempt, given your experience with easy-escape. Please forgive me for not pursuing your idea myself; although it has merit, I remain skeptical for two reasons:
First, in contrast to easy-escape, xr is an expanding substitution, sometimes strongly so, and would mess up the code layout, unless care is taken to reformat.
Second, I want to encourage people to actually use rx instead of string regexps, not just paint it over. Your proposal reminds me too much of the current dystopian trajectory of humanity, where we will be living in a polluted wasteland, all wearing AR goggles providing the illusion of an arcadian paradise.


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Mattias Engdegård-2
2 feb. 2019 kl. 10.43 skrev Mattias Engdegård <[hidden email]>:
>
> 1 feb. 2019 kl. 19.41 skrev Clément Pit-Claudel <[hidden email]>:
>>
>> I would suggest neither; instead, it would convert only strings that contain regexp constructs (these are the ones that are hard to read in the first place).
>> That;'s more or less what I do in https://github.com/cpitclaudel/easy-escape, and it works OK.

I realise that my reply looked like condescending nonsense - sorry. Please let me try again:

Thank you, that's an interesting idea! I don't think I'll work on it right now, but why don't you give it a go, using xr as the backend?



Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Clément Pit-Claudel
On 02/02/2019 10.15, Mattias Engdegård wrote:
> Thank you, that's an interesting idea! I don't think I'll work on it right now, but why don't you give it a go, using xr as the backend?

I've attached an implementation and screenshots.  rx seems to work nicely!

The expansion issue is indeed a bit of a problem.  It's also why I don't use rx often, in fact.
Another issue is regular expressions built with concat, since the parts they are made of are typically not
The last issue is that the code I posted doesn't reveal the underlying expression (i.e. doesn't remove the display property) when the point is on it, which would be needed for a robust implementation.

By the way, could forms like "[A-Za-z]" be converted to a more compact representation? At the moment xr produces (any "A-Z" "a-z"), but it seems that (any "A-Za-z") would work as well and be shorter.

Again, nice work!
Clément.

xr-prettify.el (3K) Download Attachment
rx.png (99K) Download Attachment
re.png (89K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Mattias Engdegård-2
2 feb. 2019 kl. 19.41 skrev Clément Pit-Claudel <[hidden email]>:
>
> The expansion issue is indeed a bit of a problem.  It's also why I don't use rx often, in fact.

The verbosity of rx is very much a question of habituation. It is to me a very small price to pay for its numerous advantages.
I recommend reading Olin Shivers's original SRE text [https://scsh.net/docu/post/sre.html] for a well-reasoned rationale.

> By the way, could forms like "[A-Za-z]" be converted to a more compact representation? At the moment xr produces (any "A-Z" "a-z"), but it seems that (any "A-Za-z") would work as well and be shorter.

That's a close call, but I tried it and it's probably no less readable so I made the change. Thank you.


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Juri Linkov-2
In reply to this post by Clément Pit-Claudel
> I've attached an implementation and screenshots.  rx seems to work nicely!

It also could help to find bugs, e.g. in your example the symbol ‘nonl’
inside “loaddefs.el” indicates it should be a literal ‘.’.

Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Mattias Engdegård-2
3 feb. 2019 kl. 21.11 skrev Juri Linkov <[hidden email]>:
>
> It also could help to find bugs, e.g. in your example the symbol ‘nonl’
> inside “loaddefs.el” indicates it should be a literal ‘.’.

Very eagled-eyed! Perhaps someone would fix it right away?

For that matter, I have had the same experience: bugs that hide well in a jungle of leaning toothpicks are deprived of their natural camouflage when converted to rx. Some inefficiencies also tend to become visible. The first things to go are usually the useless capture groups.


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Clément Pit-Claudel
In reply to this post by Juri Linkov-2
On 03/02/2019 15.11, Juri Linkov wrote:
>> I've attached an implementation and screenshots.  rx seems to work nicely!
>
> It also could help to find bugs, e.g. in your example the symbol ‘nonl’
> inside “loaddefs.el” indicates it should be a literal ‘.’.

Indeed! Good catch :)


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Stefan Monnier
In reply to this post by Mattias Engdegård-2
> This is a proposal to add xr to ELPA.

Done, thank you.
It's now in the `externals/xr` branch in elpa.git and the corresponding
1.0 release package should appear in GNU ELPA shortly.

As always with elpa.git, this does not automatically track remote
branches, so you'll need to `git push` to that branch when necessary.


        Stefan


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Michael Heerdegen
Stefan Monnier <[hidden email]> writes:

> 1.0 release package should appear in GNU ELPA shortly.

It has appeared.

Mattias, a question about `xr-pp':

(1) The docstring says "This basically does `(pp (rx RE-STRING))'
[...]", but there seems to be a typo, it should say "xr" instead of
"rx", right?

(2) Could `xr--pp-rx-to-str' be called `xr-pp-rx-to-str'?  I think it's
a too important and useful function to name it with a double hyphen.


Thanks,

Michael.

Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Michael Heerdegen
In reply to this post by Stefan Monnier
Stefan Monnier <[hidden email]> writes:

> > This is a proposal to add xr to ELPA.
>
> Done, thank you.

BTW, did someone notice that the functionality of "pcre2el" in GNU Elpa
overlaps with the newly added package?

Michael.

Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Mattias Engdegård-2
In reply to this post by Stefan Monnier
5 feb. 2019 kl. 17.15 skrev Stefan Monnier <[hidden email]>:

> Done, thank you.
> It's now in the `externals/xr` branch in elpa.git and the corresponding
> 1.0 release package should appear in GNU ELPA shortly.

Thanks. You were unhappy with the tests, and rightly so; I've moved them to a separate file and now use ert.

> As always with elpa.git, this does not automatically track remote
> branches, so you'll need to `git push` to that branch when necessary.

I have no push access to elpa.git. Should I request it?
The source could live directly in elpa.git instead of being an external package as far as I am concerned. Do you have any preference?

Regarding the possible mail loop: I haven't observed it before. If it keeps causing trouble, I'll investigate.


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Mattias Engdegård-2
In reply to this post by Michael Heerdegen
5 feb. 2019 kl. 23.37 skrev Michael Heerdegen <[hidden email]>:
>
> (1) The docstring says "This basically does `(pp (rx RE-STRING))'
> [...]", but there seems to be a typo, it should say "xr" instead of
> "rx", right?

Thanks, now fixed.

> (2) Could `xr--pp-rx-to-str' be called `xr-pp-rx-to-str'?  I think it's
> a too important and useful function to name it with a double hyphen.

Certainly; done.


Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Stefan Monnier
In reply to this post by Michael Heerdegen
> BTW, did someone notice that the functionality of "pcre2el" in GNU Elpa
> overlaps with the newly added package?

I missed that.  Could you mention it in the Commentary (I already added
a mention of lex-parse-re which also overlaps)?


        Stefan

Reply | Threaded
Open this post in threaded view
|

Re: [ELPA] New package: xr

Stefan Monnier
In reply to this post by Mattias Engdegård-2
> I have no push access to elpa.git. Should I request it?

If you intend to keep actively maintaining the package, then it would
make sense, yes.

> The source could live directly in elpa.git instead of being an external
> package as far as I am concerned.  Do you have any preference?

If the elpa.git repository can be the official "upstream" then it avoids
the need to keep it in sync, so that's what I recommend, but it's up to you.

> Regarding the possible mail loop: I haven't observed it before.
> If it keeps causing trouble, I'll investigate.

I'll let you know if it re-occurs.


        Stefan

123