Another computer question...

OP
Black Hole

Black Hole

May contain traces of nut
I see, you're invoking another bash session inside the loop. Can that command be run on a command line, or does it have to be in a script file?

There are loads of %whatevers you can do these days to stop you making the mess in the first place.
Interesting. This stuff post-dates when I was actively writing .bat files, so I'm not really aware of it. We used to have to install third-party command extension packages, which you could never assume would be present on another system so I didn't use them.
 

prpr

Well-Known Member
I see, you're invoking another bash session inside the loop.
Only to make the redirection work. There's bound to be at least one different way of doing it (which may or may not be better).
Can that command be run on a command line, or does it have to be in a script file?
Either.
We used to have to install third-party command extension packages, which you could never assume would be present on another system so I didn't use them.
Indeed, but M$ finally woke up and implemented some more useful stuff.
 

af123

Administrator
Staff member
This should be fine up to a certain number of files:
Bash:
for f in *.txt; do
    cat "$f" > "$f.out"
done

Otherwise you need something like
Bash:
find . -name '*.txt' -type f | while read f; do
    cat "$f" > "$f.out"
done
 
OP
Black Hole

Black Hole

May contain traces of nut
This should be fine up to a certain number of files:
Bash:
for f in *.txt; do
    cat "$f" > "$f.out"
done
That seems to be missing a semicolon before the done.

Edit: Aha! Maybe not. Is it the case that a line break implies a ;?
 

prpr

Well-Known Member
Yes. I just discovered this effect too, if you type in the following:
Code:
prpr@mint ~ $ for f in *.exe
> do echo $f
> done
vncviewer.exe
and then press the Up-arrow key you get it automagically converted:
Code:
prpr@mint ~ $ for f in *.exe; do echo $f; done
 
OP
Black Hole

Black Hole

May contain traces of nut
For the record:

I was looking for a way to get a hex dump of a file into a text file or output to printer; the Notepad++ hex extension displays a hex dump but damned if I can find a way to grab/print it (except as a screen shot). MS-DOS used to have a hex editor (DISKEDIT?) but this doesn't seem to be available as a Windows terminal command. Googling produced this:

Code:
certutil -encodehex <input> <output>
 
OP
Black Hole

Black Hole

May contain traces of nut
And the reason the above is relevant to me...

I need a way of extracting the Genre ID3 tag from a .mp3 to stdout on the command line, something like:
Code:
> id3genre test.mp3
Country
>

I've looked at loads of utilities available online, but they all have wrinkles, or would require me to extract the output I want from a messier output.

I don't need Rolls Royce, all the input .mp3s conform to the same "simple" tag structure (all generated by the same software, no quirks), with the tags at the start of the file (not the end!), so the algorithm goes something like this:
  1. Check the file starts with ASCII byte sequence "ID3";
  2. Locate first occurrence of ASCII byte sequence "TCON";
  3. Read a string character count n;
  4. At a fixed offset, read n characters (16-bit Unicode, but all ASCII);
  5. Convert to ASCII and output to StdOut.
If I were doing this I would start with Awk, but I reckon it should be a piece of piss to anyone up to speed with C (or something) to knock up id3genre.exe.

Any takers? I'll supply a sample file if so.

Detailed spec (updated 31/1/22 to account for optional ID3 padding, see 5.4):
  1. Read 10 bytes from input to array H[1:10]
  2. If H[1:3] != "ID3" (49 44 33) exit
  3. Count1 = (((H[7]&127)*128+(H[8]&127))*128+(H[9]&127))*128+(H[10]&127)
  4. Set array G = empty
  5. While Count1 ≥︎ 10 do
    1. Read 10 bytes from input to array H[1:10]
    2. Count1 = Count1 - 10
    3. Count2 = ((H[5]*256+H[6])*256+H[7])*256+H[8]
    4. If H[1:4] = [00,00,00,00] do
      1. Count1 = 0
    5. ElseIf H[1:4] = "TCON" (54 43 4F 4E) do
      1. Read Count2 bytes from input to array G
      2. Count1 = 0
    6. Else
      1. Read Count2 bytes from input and discard
      2. Count1 = Count1 - Count2
    7. Endif
  6. Loop
  7. Index = 4
  8. While G[Index] exists do
    1. Output G[Index]
    2. Index = Index + 2
  9. Loop
  10. Output End-of-Line (CRLF)
 
Last edited:
OP
Black Hole

Black Hole

May contain traces of nut
To report back: I got very close in Awk but it fell at the last hurdle. Awk is orientated to text files, so deals in ASCII character codes, but I managed to make it read a byte at a time (instead of a delimited string) by doing everything in the BEGIN (pre-processing) section, and then n-byte arrays (much of the data I'm trying to parse is straight ASCII, but some is an ASCII value in a 16-bit word [UTF-16?], and record lengths are held as binary counts). I had something very similar to the spec above implemented, and it was spitting out Genres... but not in every case.

When I looked into it (with a lot of debugging hooks), I found the failed cases were corrupting the input data. Specific byte values in the record length fields trashed my hoodwinked byte-read process, resulting in the subsequent bytes to be copies and not new data. I could find no rationale for this, except to write it off as a product of the exact implementation of Awk and noting that I was trying to use Awk outside its intended application.

I started looking at C, but couldn't find a simple straight-forward (and free) complier to settle on which didn't obviously have baggage. It was too time consuming to decide on a compiler (for Windows), let alone learn the programming syntax... and I had a "dirty" solution to fall back on: the Windows programme MP3Tag (which also works in Wine, by the way) can export user-configurable CSV tables of metadata from your MP3 (and other) audio library, so I just exported the filename and Genre tags as a table, and then used the table as input instead of extracting the genre on-the-fly.

Not the ideal solution, because I can't find a way to automate MP3Tag, but it got the job done and I won't need to do it too often.
 
Last edited:
OP
Black Hole

Black Hole

May contain traces of nut
Today's conundrum: using a Lightning-to-USB adapter, I can "see" files on a powered external HDD from my iPad... but not write to it. Is that because iOS (like Humax) can't write to NTFS?
 

EEPhil

Number 28
Oops!
I did have a look into this, based on your spec above, and came up with a not-very-forgiving C program. It (should) crap out quietly when it reads garbage (technical term in computing, not me slipping into Americanisms for rubbish). It isn't necessarily completely idiot proof. (I suppose that depends on whether you are a complete idiot :);)). The problem is/was that I only have a 32 bit machine and a version of Borland C++ that is ancient. I'm not sure whether it has compiled as a standalone executable (I think so) or whether it needs the run-time libraries (don't think so but not 100% sure). With no guarantee or warranty that it will work on your computer or meet your exacting needs here is a zip file containing my attempt. There is a text file included with ideas for a batch file and a sample output (note references to ASCII and Unicode are for my benefit). If it works, it works. If it doesn't, hey ho!
Simple instructions: From a windows command line - id3genre "filename"
Quotes are important if there are any spaces or other odd characters in the filename. Should just output the genre on the next line, or an empty line if no genre found in the right place or an error occurred. (fingers crossed).
 

Attachments

  • id3genre.zip
    33.5 KB · Views: 5

EEPhil

Number 28
Today's conundrum: using a Lightning-to-USB adapter, I can "see" files on a powered external HDD from my iPad... but not write to it. Is that because iOS (like Humax) can't write to NTFS?
Huh! I have the reverse problem (ish). My Windows computer can see files on the iPhone when connected with one of those lightning cables. I can read them, but cannot write to the iPhone. I think I need iTunes to do that. But the most recent version wont run on the PC and the old version wont connect to the iPhone. Have to log in to iCloud and send files that way. Argh!
 
OP
Black Hole

Black Hole

May contain traces of nut
Huh! I have the reverse problem (ish). My Windows computer can see files on the iPhone when connected with one of those lightning cables. I can read them, but cannot write to the iPhone. I think I need iTunes to do that. But the most recent version wont run on the PC and the old version wont connect to the iPhone. Have to log in to iCloud and send files that way. Argh!
I use the "FE File Explorer" app, and move files around by SMB. FE can see into Files and Camera Roll, connect to any network storage available, and even USB (Lightning) connected storage. As above, it won't write to the (NTFS) USB drive though.

The reason I want to write by USB drive is for speed to make a bulk copy of my camera roll.
 

bottletop

Active Member
I use the "FE File Explorer" app, and move files around by SMB. FE can see into Files and Camera Roll, connect to any network storage available, and even USB (Lightning) connected storage. As above, it won't write to the (NTFS) USB drive though.

The reason I want to write by USB drive is for speed to make a bulk copy of my camera roll.
Maybe try using reformatting drive to FAT32 or ExFAT?
FAT32 is older and may have better support, but has 4GB file size limit. As it is for camera roll - that may be ok.
ExFAT is newer so may be less widespread use but does not have the 4GB file size limitation.
 
Last edited:

Allotment

New Member
To report back: I got very close in Awk ............................... etc


I also looked a this and compiled a shell around a Go language library. Same instructions as those provided by eePhil. Outputs a genre or an error message (I can suppress the error if needed)
 

Attachments

  • id3genre.zip
    754.1 KB · Views: 4
OP
Black Hole

Black Hole

May contain traces of nut
Maybe try using reformatting drive to FAT32 or ExFAT?
:rolling:
Non-destructively?

FAT32 is older and may have better support, but has 4GB file size limit.
I presume that's for the benefit of other readers.


I also looked a this and compiled a shell around a Go language library.
Super, thanks. Pity I didn't have these offerings a couple of months ago, but at least I learned a lot about twisting Awk and the limitations thereof!
 

EEPhil

Number 28
Are you aware of the Embarcadero C++ Community Edition which is free; see https://www.embarcadero.com/products/cbuilder/starter
I am aware of that. I think I did look into it ages ago but I couldn’t see any advantage over the Borland C++ Builder I have. I will have another look, thanks.

Edit: Ah, yes. Wrong version of Windows, not enough memory... Perhaps if/when I get a new computer with Windows 10 or higher.
 
Last edited:
Top