tag editing

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

tag editing

Grant Shoshin Shangreaux
Hello all,

working on getting my FSF assignment finished and looking forward to
contributing to the project soon.

I bumped into an issue when trying to edit tags for opus files, it
looked like it all worked fine, saved the metadata to the cache-db, but
there was no actual function defined to handle the opus format. so it
was a surprise to me to find that the files i tagged no longer had the
metadata the when i opened them in another program.

i'd like to fix that silent failure, but i also started digging around
for a solution for writing tags to all formats. unfortunately, there
isn't a simple tag writer included with the opus-tools package. you must
re-encode the audio to add tags. so far, i've found 3 candidates

  1. add a shim for TagLib to write to files (beyond me at the moment)
  2. opuscomment[1] tool (must be built from source)
  3. audiotools package's tracktag program[2]

I wrote a simple wrapper for tracktag that hooks into the tag-editor
code and have been testing that out. I'd be happy to share a patch, but
I can wait until the FSF assignment is done if that is better.

Another option might be to hook into the emms-info-native code and add
tag writing capabilities as well. I would be happy to collaborate on
that, but my time is somewhat limited. I do want to point out that
native tag writing capality would be really fantastic for my use of
EMMS. I record a lot of audio and have been hoping for a nice way to
author metadata using Emacs. I even gave a talk at emacs-conf demoing
how nice it was to use EMMS[3], not knowing my opus tags weren't even
being written xD.

Anyhow, please let me know if there's interest in the tracktag code, and
if there's any good way to help out otherwise.

best,
Grant Shangreaux


[1] https://github.com/hcmiya/opuscomment
[2] http://audiotools.sourceforge.net/tracktag.html
[3] https://emacsconf.org/2020/talks/05/

Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Yoni Rabkin-2
Grant Shoshin Shangreaux <[hidden email]> writes:

> Hello all,
>
> working on getting my FSF assignment finished and looking forward to
> contributing to the project soon.

That is good to hear; thank you.

> I bumped into an issue when trying to edit tags for opus files, it
> looked like it all worked fine, saved the metadata to the cache-db, but
> there was no actual function defined to handle the opus format. so it
> was a surprise to me to find that the files i tagged no longer had the
> metadata the when i opened them in another program.
>
> i'd like to fix that silent failure,

This is a priority and would form a basis for all further tag editing
work. It would be great if you started there.

We would want that, if the tag editor is calling an executable, that a
failed process call would fail loudly.

Then we want that regardless of how the tags are being written, natively
or with an external process, that the editor would check if the tag had
indeed been updated in the file by comparing the tags in the file before
and after update. Trust, But Verify.

> but i also started digging around for a solution for writing tags to
> all formats. unfortunately, there isn't a simple tag writer included
> with the opus-tools package. you must re-encode the audio to add
> tags. so far, i've found 3 candidates

Re-encoding is a no-go.

>   1. add a shim for TagLib to write to files (beyond me at the moment)
>   2. opuscomment[1] tool (must be built from source)
>   3. audiotools package's tracktag program[2]

Packages such as audiotools are themselves abstractions to C libraries
such as libmpg123, libdvd-audio, etc. (like tinytag, which Emms calls).

We likely aren't going to ask to have any of those C libraries be linked
to Emacs on compilation, so the best scenario is indeed native reading
and writing, and that is a worthy goal.


--
   "Cut your own wood and it will warm you twice"

Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Petteri Hintsanen-2
In reply to this post by Grant Shoshin Shangreaux
Grant Shoshin Shangreaux <[hidden email]> writes:

> Another option might be to hook into the emms-info-native code and add
> tag writing capabilities as well.

I wouldn’t recommend going this way, at least not yet.  Some rationale:

1. There are already plenty of solid free implementations for writing
   metadata, so this would be reinventing the wheel, and probably badly
   in comparison.

2. Writing to a file always carries the risk of messing up that file.
   Users would better have backups of their precious audio files.  This
   is true, of course, for all tag writing software, but especially so
   for a new implementation like this.

3. Adding support for writing tags is not straightforward as
   emms-info-native tends to cut corners due to its read-only nature.
   For example, it occasionally throws data away while decoding,
   including tags that EMMS does not recognize.  To prevent data loss,
   this ought to be fixed if the data would be written back at some
   point.

   Also, emms-info-native does not work with files that have multiple
   (multiplexed) streams.  It is questionable how many audio files do
   have more than one logical stream, but such files should be detected
   and any failures handled gracefully.

4. I’m looking forward to add MP3 (id3v2) support for emms-info-native
   but not more.  Support for MP4, for instance, is pretty complex to
   do.  With external tools we get much wider coverage for popular
   formats.  I personally don’t care for nonfree formats, but others
   may.

Personally I like the lightweight EMMS design that emphasizes the use of
external programs instead of (re)implementing stuff by itself.
Consequently, I see emms-info-native more like a simplistic stopgap
solution for those (hopefully rare) use cases where there are no
external metadata tools available or configured.  For any serious
metadata operations I would go with external tools.  In my own setups I
use emms-print-metadata and MPD, and MusicBrainz Picard for tagging.

This does not mean that emms-info-native couldn’t be a "simplistic
stopgap solution" for writing tags as well, but please consider the
caveats listed above.

Speaking of solutions,

> 1. add a shim for TagLib to write to files (beyond me at the moment)

this one is probably easy to do.  The current Taglib shim
(emms-print-metadata) can be extended to support tag updates as well.
I can help with that if needed.

Thanks,
Petteri

Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Grant Shoshin Shangreaux

Petteri Hintsanen writes:

> I wouldn’t recommend going this way, at least not yet.

your insight is very much appreciated. i think all the points you
brought up are really important to consider.

> 1. There are already plenty of solid free implementations for writing
>    metadata

agreed, though I've not found a single executable that handles all the
formats and metadata at once. i think TagLib has the capability, and
audiotools tracktag is pretty close. oddly, tracktag does not write a
"genre" tag, which is one of the standard emms-info items.

> 2. Writing to a file always carries the risk of messing up that file.

yes, would be very sad to have EMMS destroy audio files. especially the
ones i'm working with which are originals. i often have backups, but
this is very important. i certainly don't want to be responsible for the
code that is doing it.

> 4. I’m looking forward to add MP3 (id3v2) support for emms-info-native
>    but not more.  Support for MP4, for instance, is pretty complex to
>    do.  With external tools we get much wider coverage for popular
>    formats.  I personally don’t care for nonfree formats, but others
>    may.

Another reason I started working with tracktag. I only cared about
adding metadata to opus, ogg, and flac, but having one interface that
can handle the other formats is a win for wider use.

> For any serious metadata operations I would go with external tools.
> In my own setups I use emms-print-metadata and MPD, and MusicBrainz
> Picard for tagging.

i agree. i had a sense that empowering emacs to do it would avoid some
of the hassle of setting up the external tools, but its really not that
hard. I built emms-print-metadata not realizing that it didn't support
tagging. i tried for a bit to work on the Taglib shim, but my lack of
time to hack and experience with C was blocking me.

>> 1. add a shim for TagLib to write to files (beyond me at the moment)
>
> this one is probably easy to do.  The current Taglib shim
> (emms-print-metadata) can be extended to support tag updates as well.
> I can help with that if needed.

That would be fantastic. I can continue with tracktag as well, it might
be useful since it is installable with a package manager for users who
might not go the length of compiling the emms-print/write-metadata
helpers. no rush on this, i've solved my immediate problem of writing
tags to opus files, but would be glad to contribute to any wider
solution!

Thanks,
Grant Shangreaux


Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Yoni Rabkin-2
In reply to this post by Petteri Hintsanen-2

Petteri Hintsanen <[hidden email]> writes:

> 4. I’m looking forward to add MP3 (id3v2) support for emms-info-native
> but not more.  Support for MP4, for instance, is pretty complex to do.
> With external tools we get much wider coverage for popular formats.  I
> personally don’t care for nonfree formats, but others may.

Don't worry; we won't be adding support for non-free formats during my
tenure as maintainer.

--
   "Cut your own wood and it will warm you twice"

Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Petteri Hintsanen-2
In reply to this post by Grant Shoshin Shangreaux
Grant Shoshin Shangreaux <[hidden email]> writes:

>>> 1. add a shim for TagLib to write to files (beyond me at the moment)
>>
>> this one is probably easy to do.  The current Taglib shim
>> (emms-print-metadata) can be extended to support tag updates as well.
>> I can help with that if needed.
>
> That would be fantastic. I can continue with tracktag as well, it might
> be useful since it is installable with a package manager for users who
> might not go the length of compiling the emms-print/write-metadata
> helpers. no rush on this, i've solved my immediate problem of writing
> tags to opus files, but would be glad to contribute to any wider
> solution!

Yes I also agree that something that is available from distribution
repositories is preferable.  Building and installing emms-print-metadata
with its dependencies is not exactly easy for novices, though it is not
rocket science either.

Just let me know if you decide to go with taglib and need help with it.  
It is mostly just a matter of defining the interface between the shim
and EMMS. Actual C++ implementation should be straightforward.

thanks,
Petteri

Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Grant Shoshin Shangreaux

Petteri Hintsanen writes:

> Yes I also agree that something that is available from distribution
> repositories is preferable.  Building and installing emms-print-metadata
> with its dependencies is not exactly easy for novices, though it is not
> rocket science either.
>
> Just let me know if you decide to go with taglib and need help with it.  
> It is mostly just a matter of defining the interface between the shim
> and EMMS. Actual C++ implementation should be straightforward.

So far I've only worked on the audiotools tracktag integration. i've got
it working in a very basic way, i'm just still waiting to get my work to
sign off on the FSF assignment before sending any patches.

I think taglib could be useful in some cases, but not a priority for me
as tracktag is enabling me to tag the opus files i was having trouble
with.

thanks for offering help! i'm definitely interested in learning how to
do it, i just haven't worked with C++ projects before.

--Grant

Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Grant Shoshin Shangreaux
In reply to this post by Yoni Rabkin-2

>> working on getting my FSF assignment finished and looking forward to
>> contributing to the project soon.
>
> That is good to hear; thank you.

sorry about the delay, it took quite a while to get the attention of my
employer to finish the process. i truly hope i'll be clear before the
next emms release, as i have a few minor contributions i hope to provide.

>> the files i tagged no longer had the
>> metadata the when i opened them in another program.
>>
>> i'd like to fix that silent failure,
>
> This is a priority and would form a basis for all further tag editing
> work. It would be great if you started there.
>
> We would want that, if the tag editor is calling an executable, that a
> failed process call would fail loudly.
>
> Then we want that regardless of how the tags are being written, natively
> or with an external process, that the editor would check if the tag had
> indeed been updated in the file by comparing the tags in the file before
> and after update. Trust, But Verify.

i've done some work around this issue, and it looks like rather than a
failed executable process, the problem is that there is no tagfile
function configured for the Opus file type. only mp3, ogg, and flac are
supported. EMMS just skips trying to write to the files, but updates the
cache with no problem and the only message a user sees is "Setting
tags...done".

my proposed solution is to warn the user before even loading metadata
into the tag editor. if a single track is selected, trying to edit an
unsupported format (no match in the emms-tag-editor-tagfile-functions
variable) will NOT open the tag editing buffer and instead issue a
warning message that there is no tag writing function configured. this
can be overridden with the universal argument if a user wants to edit
tags anyway, knowing that at least EMMS can save them in the cache. if
the track is not a file, it allows the user to edit metadata regardless
(for example, you want to add data to a url track). if multiple tracks
are marked and you try to edit the metadata, any unsupported files
will not appear in the tag editor buffer and a warning message will
appear. the universal argument again will allow tag editing anyway.

what do you think about this option? its based primarly in my own
experience of the application and what i would have liked to have happen
as i was tagging a handful of opus files.

i've yet to handle cases where the executable fails, as is i think it
only puts a message into the EMMS log. i'm not sure what the "best" way
to handle this is, especially when writing to multiple files. i'll
continue working on this

i also still need to do the verification of tag updates after it
happens. i'm working on writing up some ert tests to make it a bit
easier to ensure, my plan was to generate some test audio files for the
tag editor to operate on. i'm not sure if its practical to set up
automated testing for EMMS at large, but at least on my local machine it
will help me progress on this. please let me know if you have any
thoughts about it :)

>> but i also started digging around for a solution for writing tags to
>> all formats. unfortunately, there isn't a simple tag writer included
>> with the opus-tools package. you must re-encode the audio to add
>> tags. so far, i've found 3 candidates
>
> Re-encoding is a no-go.
>
>>   1. add a shim for TagLib to write to files (beyond me at the moment)
>>   2. opuscomment[1] tool (must be built from source)
>>   3. audiotools package's tracktag program[2]
>
> Packages such as audiotools are themselves abstractions to C libraries
> such as libmpg123, libdvd-audio, etc. (like tinytag, which Emms calls).
>
> We likely aren't going to ask to have any of those C libraries be linked
> to Emacs on compilation, so the best scenario is indeed native reading
> and writing, and that is a worthy goal.

i've been testing out my wrapper for tracktag with decent success so
far. i hope to also contribute that to EMMS as it provides a "one tool
fits all" tag writer that can be installed by a package manager. i still
like the idea of a TagLib shim, but for the time being this works for
me. Petteri suggested that native writing may be a much bigger job than
is worth it for Elisp, though that would be ideal (assuming it can
manage to update precious files without error :) )

Thanks for all the work on the project, once again I hope soon to make
some real contributions!

- Grant

Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Yoni Rabkin-2
Grant Shoshin Shangreaux <[hidden email]> writes:

>>> working on getting my FSF assignment finished and looking forward to
>>> contributing to the project soon.
>>
>> That is good to hear; thank you.
>
> sorry about the delay, it took quite a while to get the attention of my
> employer to finish the process. i truly hope i'll be clear before the
> next emms release, as i have a few minor contributions i hope to provide.

That is all par for the course, as they say. Don't worry about the
timing. We get to determine when a release comes out.

>>> the files i tagged no longer had the
>>> metadata the when i opened them in another program.
>>>
>>> i'd like to fix that silent failure,
>>
>> This is a priority and would form a basis for all further tag editing
>> work. It would be great if you started there.
>>
>> We would want that, if the tag editor is calling an executable, that a
>> failed process call would fail loudly.
>>
>> Then we want that regardless of how the tags are being written, natively
>> or with an external process, that the editor would check if the tag had
>> indeed been updated in the file by comparing the tags in the file before
>> and after update. Trust, But Verify.
>
> i've done some work around this issue, and it looks like rather than a
> failed executable process, the problem is that there is no tagfile
> function configured for the Opus file type. only mp3, ogg, and flac are
> supported. EMMS just skips trying to write to the files, but updates the
> cache with no problem and the only message a user sees is "Setting
> tags...done".
>
> my proposed solution is to warn the user before even loading metadata
> into the tag editor. if a single track is selected, trying to edit an
> unsupported format (no match in the emms-tag-editor-tagfile-functions
> variable) will NOT open the tag editing buffer and instead issue a
> warning message that there is no tag writing function configured. this
> can be overridden with the universal argument if a user wants to edit
> tags anyway, knowing that at least EMMS can save them in the cache. if
> the track is not a file, it allows the user to edit metadata regardless
> (for example, you want to add data to a url track). if multiple tracks
> are marked and you try to edit the metadata, any unsupported files
> will not appear in the tag editor buffer and a warning message will
> appear. the universal argument again will allow tag editing anyway.
>
> what do you think about this option? its based primarly in my own
> experience of the application and what i would have liked to have happen
> as i was tagging a handful of opus files.

This is a good thing to do, and I would like to see it
implemented. However, it doesn't replace validation (more on this
below.)

> i've yet to handle cases where the executable fails, as is i think it
> only puts a message into the EMMS log. i'm not sure what the "best" way
> to handle this is, especially when writing to multiple files. i'll
> continue working on this

There are three levels of error to handle: first is when `call-process'
cannot find an executable, second is when the executable ran but
returned an error (redirecting and looking for output in the
standard-error stream may help here in well behaved executables), and
finally when the executable ran without error but didn't do the job we
wanted. This final one is addressed below.

> i also still need to do the verification of tag updates after it
> happens. i'm working on writing up some ert tests to make it a bit
> easier to ensure, my plan was to generate some test audio files for
> the tag editor to operate on. i'm not sure if its practical to set up
> automated testing for EMMS at large, but at least on my local machine
> it will help me progress on this. please let me know if you have any
> thoughts about it :)

Validation is important for Emms to become dependable as a tag
editor. But don't worry overly much about creating robust testing; if
you are doing all of the development yourself you can, and should, rely
on the rest of us to hammer your code. Send it out to a branch on the
repo and ask us to throw stuff at it.

>>> but i also started digging around for a solution for writing tags to
>>> all formats. unfortunately, there isn't a simple tag writer included
>>> with the opus-tools package. you must re-encode the audio to add
>>> tags. so far, i've found 3 candidates
>>
>> Re-encoding is a no-go.
>>
>>>   1. add a shim for TagLib to write to files (beyond me at the moment)
>>>   2. opuscomment[1] tool (must be built from source)
>>>   3. audiotools package's tracktag program[2]
>>
>> Packages such as audiotools are themselves abstractions to C libraries
>> such as libmpg123, libdvd-audio, etc. (like tinytag, which Emms calls).
>>
>> We likely aren't going to ask to have any of those C libraries be linked
>> to Emacs on compilation, so the best scenario is indeed native reading
>> and writing, and that is a worthy goal.
>
> i've been testing out my wrapper for tracktag with decent success so
> far. i hope to also contribute that to EMMS as it provides a "one tool
> fits all" tag writer that can be installed by a package manager. i still
> like the idea of a TagLib shim, but for the time being this works for
> me. Petteri suggested that native writing may be a much bigger job than
> is worth it for Elisp, though that would be ideal (assuming it can
> manage to update precious files without error :) )

I'd take Petteri's word on this.

> Thanks for all the work on the project, once again I hope soon to make
> some real contributions!

Looking forward to it.

--
   "Cut your own wood and it will warm you twice"

Reply | Threaded
Open this post in threaded view
|

Re: tag editing

Petteri Hintsanen-2
In reply to this post by Grant Shoshin Shangreaux
Grant Shoshin Shangreaux <[hidden email]> writes:

> i also still need to do the verification of tag updates after it
> happens. i'm working on writing up some ert tests to make it a bit
> easier to ensure, my plan was to generate some test audio files for the
> tag editor to operate on. i'm not sure if its practical to set up
> automated testing for EMMS at large, but at least on my local machine it
> will help me progress on this. please let me know if you have any
> thoughts about it :)

ERT tests are good idea I think.  I'm also planning to wrap at least
some of the tests I've used for testing emms-info-native into ERT test
cases.  The only real issue is copyright: all test data must have a
license that is GPL compatible.  I've used data from Mutagen
(https://github.com/quodlibet/mutagen) which is ok as it is under GPLv3,
but I still have to check other files.

> me. Petteri suggested that native writing may be a much bigger job than
> is worth it for Elisp, though that would be ideal (assuming it can
> manage to update precious files without error :) )

It is quite a job indeed, and the main culprit is MP3.  id3v2 tags have
lots of complications--even within a single id3v2.x spec version--so I'm
not that enthusiastic about writing them.

Vorbis-like comments are much easier so native writing for Ogg/FLAC/Opus
is probably feasible.  It would require some redesign of
emms-info-native but not prohibitively much.

I would still recommend to continue on your chosen track as it will
improve the state of the art, and (I think) we should have robust
support for external metadata writing tools.  I also think that it is
good idea to go ahead one step at a time; that is, first make sure that
the chosen tagger does not exit with error, and after that works,
validate the output like Yoni suggested.  Even the first step without
the second is better than the current state.

Thanks,
Petteri