Solved, see below.

I recently reinstalled my home server and was unable to open my LUKS-encrypted hard drive. Neither my usual passphrase nor a newly created keyfile were working. I tested on different distros, initially on my new Proxmox installation, later on a the Arch ISO. I eventually tried the disk on my main system, on which it used to be and I still had an old keyfile on - et voilá. So I created keyfiles as suggested in the wiki and occasionally md5sum returned a different hash for the keyfile! Why is this happening? I find’ this extremely concerning because this could potentially result in massive data loss due to a keyfile apparently randomly not working as I was experiencing it. What am I missing?

For reference because I don’t know how to share what I exactly did.

Scenario #1:
A directory on a mounted hard drive on my desktop.

$ echo -n '$mypassphrase' > ./dir/keyfile
$ md5sum ./dir/keyfile
 c6dd9329dbe030127ce5e19d85de4df9 ./dir/keyfile
# chown root:root ./dir/keyfile; chmod 400 ./dir/keyfile
# md5sum ./dir/keyfile
c6dd9329dbe030127ce5e19d85de4df9 ./dir/keyfile

Scenario #2:
My old keyfile in /etc on my desktop containing $mypassphrase.

# md5sum /etc/keyfile
a1c10c2d023c982259f6c945ebee664e /etc/keyfile

Scenario #3:
Booted from the Arch ISO on my server.

# echo -n '$mypassphrase' > keyfile
# chown root:root keyfile; chmod 400 keyfile
# md5sum keyfile
c6dd9329dbe030127ce5e19d85de4df9 keyfile

Scenario #4:
A directory in /home on my desktop.

$ echo -n '$mypassphrase' > ./keyfile
$ md5sum keyfile
a1c10c2d023c982259f6c945ebee664e keyfile

EDIT: I just moved the disk back into my server and tried echo’ing my passphrase into a keyfile which returned the hash starting with c6, whereas opening a file using nano and pasting the passphrase into the file returned the hash starting with a1.

EDIT: I moved the disk back into my server, reinstalled Proxmox and tried again. I was able to unlock the disk after I pasted the passphrase into a file and deleted all trailing spaces/newline. I also tried echo’ing the passphrase into a keyfile and that also did not work, no clue why but it seems to work on some systems on not on others.

  • axzxc1236@lemmy.world
    5·
    1 year ago

    nano by default puts a new line character at the end of file, can you use stat command to check if file size is off by 1 byte?

    • Dataprolet@lemmy.dbzer0.comOPEnglish
      1·
      1 year ago

      Thanks for your answer. I’m using UTF-8 on my desktop running Arch Linux and my server running Proxmox.

      $ echo -n 'myrandompassphrase123' > keyfile
      $ md5sum keyfile
      49b3cdce58e65935b8255438266dd87e keyfile

      Whereas editing the keyfile using nano and pasting ‘myrandompassphrase123’ returns 59be855b925013d35476e82763b7a80e.
      This is the scenario that’s the least concerning, because I get that echoing into a file and editing a file using nano could yield different results. Unfortunately (or luckily) I can’t reproduce the 4th scenario, which concerned me.

      • ThreeHalflings@lemmy.world
        1·
        1 year ago

        I’ll have a play tomorrow and see what I get.

        I suspect opening up the two files in a hex editor will show the difference, assuming I can replicate your results.

      • ThreeHalflings@lemmy.world
        1·
        1 year ago

        I suspect you can’t replicate scenario 4 because it was never a thing (Pretty natural to skip over something by accident). A trailing newline was my first thought, but scenario 4 ruled it out. If you can’t replicate it then it’s the likely answer.

        Something like this would confirm

        echo "$mypass" | md5sum

        Vs

        echo -e "$mypass\n" | md5sum