Publishing org mode notes as a website

… and the optional migration from hugo.

Over the years, I used text files, cms, trello, hugo, trello with hugo and finally org-mode to manage various kind of notes, documents and otherwise interresting to me data.

I am not a blogger, I just do my things and eventually decide to publish some of this work. Publishing is secondary and wont concern more than 5% of my notes and documents. As such, the process should be as simple as possible.

1 Convert current hugo content to org mode

Blog uses hugo, with historical contents written in markdown + JSON frontmatter for the most part.

org-mode takes org mode file, with org-mode headers.

pandoc can convert mardown to org mode, but it may need some preprocessing to handle metadata, with some limits anyway1. Not that they are so important, but I would appreciate to keep the categories, if possible.

cd ~/src/git.jardinmagique.info/jeromenerf/blog
TAILN=10
ORGDIR=content.org
YAMLDIR=content.yaml

# convert blog posts front matter to yaml, which should be handled later by pandoc to convert to org
[ -e $YAMLDIR ] && rm -fr $YAMLDIR
hugo convert toYAML -o $YAMLDIR

# Remove exsting org files from the conversion
find $YAMLDIR -name "*.org" -delete

for MDFILE in $(find $YAMLDIR/ -name "*.md"); do
    ORGFILE=${MDFILE%%.md}.org

    # preprocessing
    for md in Description Title Author Date ; do
        sed -i "s/^\($md\):/\L\1:/g" $MDFILE
    done

    # Remove the Author, if any, I don't need it.
    sed -i "/^Author:/d" $MDFILE

    # Hack to store categories temporarily in the "author" tag
    for category in photo hack craft ; do
        sed -i "s/^- \($category\)/author: \1/g" $MDFILE
    done


    # convert to org mode file format
    # standalone document, with metadata headers
    # YAML metadata blocks
    pandoc -s -f markdown+yaml_metadata_block -t org $MDFILE -o $ORGFILE

    sed -i "s/AUTHOR/TAGS/g" $ORGFILE
done

# Then copy the previous content into the new converted directory
cp -fr content $YAMLDIR/


# cleanup the remaining md files
find $YAMLDIR -name "*.md" -delete


# generating the org file structure
# get all known tags
TAGS=$(grep -r "TAGS:" $YAMLDIR | sed "s/^.*: \(.*\)/\1/g" | sort -u)

# create a clean destination
[ -e $ORGDIR ] && rm -fr $ORGDIR
mkdir -p $ORGDIR

# create a folder for every TAG
for TAG in $(echo $TAGS) ; do
    mkdir $ORGDIR/$TAG;
done

# move the posts to the correct folder
INDICES=$(find $YAMLDIR -name index.org)
for INDEX in $(echo $INDICES) ; do
    TAG=$(grep -r "TAGS:" $INDEX | sed "s/^.*: \(.*\)/\1/g" | sort -u)
    PARENT=${INDEX%index.org}
    NEWPARENT=$(basename $PARENT |sed "s/[0-9]\{2\}-[a-zA-Z0-9]\{8\}-//g" )

    # Include links to image in the posts, just in case.
    ATTACHMENTS=$(find $PARENT -type f | grep -v index.org)
    for ATTACHMENT in $(echo -n $ATTACHMENTS) ; do
        echo "[[./${ATTACHMENT#$PARENT}]]" >> $INDEX 
    done

    mv $PARENT $ORGDIR/$TAG/${NEWPARENT}
done


INDICES=$(find $YAMLDIR -name "*.org")
for INDEX in $(echo $INDICES) ; do
    TAG=$(grep -r "TAGS:" $INDEX | sed "s/^.*: \(.*\)/\1/g" | sort -u)
    mv $INDEX $ORGDIR/$TAG/
done


#mv $ORGDIR ~/org/





2 Things to setup

Better read the annotated configuration.

3 Hosting

3.1 Setup syncthing shares between locals and remote

I can share the whole ~/public_html folder on local and remote.

sbxkp-zktys

3.2 Manage permissions

OK, the 90ies again.

Static files permissions are never the same between hosts and between owner and httpd daemon.

A simple sticky bit on the ~/public_html folder should make the trick.

sudo adduser jerome www-data # if not already
chown -R jerome:www-data ~/public_html/
chmod g+rs ~/public_html/ 

4 Utilities

When testing, it might be great to trigger the build automatically.

find ~/org | entr -c -s "rm -fr ~/public_html/blog/* && emacsclient --no-wait --eval '(org-publish-all)'"

5 Resources

Footnotes: