Microblogging Journal through 11/27/2023

Dispatches from @ultrviolet@gts.endgameviable.com:

Monday 11/20

• 07:47 # It’s a joyous morn, for I have successfully gotten my recycling wheelie bin to the curb on the right day for pickup. A special recycling pickup schedule is new to me, since the county dropped off a recycling bin for everyone here in the neighborhood some months ago. They pick up every two weeks on Mondays, which is radically different from the trash pickup every Friday. I have a roughly 33% success rate of remembering to put my trash out every Friday, so remembering which Monday to put my recycling out is an even more daunting prospect. #homeowner

• 14:04 # Naturally the heat pump checker dude got here exactly at the time of day that I’ve completely run out of energy and just want to take a nap.

• 18:19 # New Post: The Sudden Urge To Rebuild The Site https://endgameviable.com/dev/2023/11/sudden-urge-to-rebuild-the-site/ (also an experiment to see what I can do with the gotosocial api)

  • 19:14 # This is a reply to that note about the blog post. I didn’t do this with curl, I just typed it into semaphore.social, but it’s still pretty impressive just the same.

  • 19:20 # This is an interesting experiment because I can pick up these notes with authorized api calls to my gotosocial server. For example, the statuses/id/context api call returns the reply thread of any status, which can just be plopped at the end of a blog post as a comment thread.

  • 18:00 # In the continuing adventures of Next.js blog experiments, I’ve just wired up a successful test of a hybrid server-and-client-side component that will render a blog post as a static page and then fetch and display Mastodon replies to that post on the client-side. The docs for Next.js aren’t terribly clear for these sorts of things. #development #nextjs #blog

  • 20:57 # Created a DynamoDB table to map blog post urls to activityPub status IDs, so I can display activityPub mentions at the bottom of a blog post. Tested some code that will detect changes in a blog’s feed and trigger notifications, which will let me create an activityPub notification post when new blog posts are created. The trigger will also update the mapping table so that the blog post page can lookup related activityPub mentions. #development #nextjs #blog

  • 20:59 # Using a mapping database keeps me from having to embed an activityPub status ID into the blog post metadata itself, which is a real pain. #development #blog

Tuesday 11/21

• 10:08 # I’m as surprised as anyone, but Lower Decks is actually pretty entertaining. It’s Star Trek The Next Gen-Z, where everything is set at 2x playback speed. Anyway it’s way more entertaining than regular Star Trek stuff, which I haven’t much cared for since the 90s.

• 13:51 # On the Next.js blog experiment front, I’m again finding myself wondering why I need this. The original thought was to mix static page generation with dynamic capabilities you can only get with a server, but what dynamic features do I actually need? Comments? Maybe, but there are comment hosting solutions, or I could even write my own, that don’t require any server-side processing. Direct ActivityPub integration was another thought, to replace comments, but that’s a fairly large chore. I dunno. Time to step back and figure out what I actually need to accomplish.

• 13:54 # Side note: One of the problems I’m wrestling with of integrating ActivityPub into a statically-built site is that there aren’t really any natural blog post publishing events, which is what would trigger broadcasting to followers. The site is always built statelessly, so it has no knowledge of what’s new, in order to trigger events.

• 18:39 # I created this note with a curl command. If this works, I should also be able to programmatically add notifications from my blog, too.

Wednesday 11/22

• 09:59 # Survived a day-before-Thanksgiving trip to the grocery store. Just needed enough stuff to make it until a-few-days-after-Thanksgiving.

• 10:02 # Working on a json feed for the blog today. Need one to test a part of the experimental Next.js Rube Goldberg blog machine (well, I don’t need one, but it’s easier to read json than xml in most modern languages). #development #blog

Thursday 11/23

• 08:37 # It’s a Thanksgiving day miracle… I discovered a new (to me) Amazon 2023 build image for AWS Amplify which has full node 18 + Next.js 14 support. No more custom node 18 build image that only builds a limited number of times a day. #development #nextjs

• 10:04 # You know, it’s just occurred to me that if I’m going to the trouble of having a Hugo build step to turn Markdown files into an S3 bucket full of json files, I might as well go a step further and put the json data files into a DynamoDB table and make things a bit easier on myself. Hrm. Can Hugo do that? Tests are required. #development #blog

  • 10:18 # After scanning the Hugo docs, the answer is: I don’t think Hugo has any functions to send payloads to remote api calls. It can only fetch data from remote api calls. Which means if I want to populate a DynamoDB table after a Hugo build, I’ll probably need to write some sort of node.js index.js application which will consume the Hugo output and upload it to DynamoDB, which I should be able to add as another build step. Tests are required. #development #blog

  • 10:33 # Hrm actually it’s probably not feasible to put a node.js application into the same repo with a Hugo content build. Annoying. Maybe I can trigger a Lambda function after the Hugo build. #development #blog

• 14:49 # Well, initial tests of putting posts into a DynamoDB aren’t very promising. The task of filling a table with 1500 items exceeds my throughput limits very quickly unless I severely rate-limit it to the point where it takes some five minutes to populate the table, which is a pretty hefty build time for a questionable gain. #development #blog

  • 14:54 # In theory I can populate the table incrementally by detecting which posts are new, instead of always rebuilding the entire table every time. Will do some tests to see if there’s any benefit to reading from DynamoDB instead of S3 first though. #development #blog

  • 20:18 # I think I’m giving up on DynamoDB… I think there’s some benefit but it’s just such a hassle to deal with the rate limits and come up with an effective query strategy. Maybe there’s another database I can try. Hrm, no there isn’t. DynamoDB looks like the only chance of staying within the aws free tier. #development #blog

  • 20:22 # One problem I’ve having is page routes that display lists of posts. It’s easy to grab one post out of a DynamoDB table with a 1:1 mapping of an url to a database key, but mapping an url to a query for the first N posts matching that path is more challenging than I might have expected. To say nothing of a simple text search. Problems for another day. #development #blog

Friday 11/24

• 13:32 # Latest experiments with storing pages in DynamoDB suggest it could be useful in implementing a search feature, but that’s about it. #development #aws #blog

• 17:18 # Hrm I am already seeing issues with CommentBox. Like today it’s just kind of sitting there, not loading. #blog

• 21:47 # Yeesh ChatGPT has been sloooooow the last couple of days. I guess this is how they protect us from the robot apocalypse. #ai

• 23:18 # I don’t really like to use nameless third-party services but I just hooked up webmention.io to my static blog. Which means now I’ll get a pingback whenever y’all retrophiles using WordPress mention my blog. It doesn’t do anything, but I’ll be able to see it on the webmention dashboard, at least. (I’m thinking about integrating a webmention api directly into the new Next.js blog, which is what led me to webmention.io.) (The new Next.js blog is still theoretical at this point.) #blog

  • 10:25 # Speaking of webmentions, I enabled pingback support in my experimental webmention.io account, and received WordPress pingback spam from a dubious site within 24 hours. Another reminder of why I’m happy I’m not part of that ecosystem anymore. #blog

Saturday 11/25

• 10:23 # It occurs to me that an AWS SQS message queue is the perfect solution for handling things like post publication notifications and/or webmentions, so I’m experimenting with an SQS queue today. (We use these things all the time in the day job, but this is the first time I’ve tried to set one up for personal use.) #development #aws #blog

• 13:24 # I’ve suddenly gotten it into my head that I want to put all this new Next.js blog code and related services into one big monorepo and also create terraform to setup the required aws resources. It’s a bit of a sprawling mess of moving parts already and I want to tame it before it gets way too far out of hand. #blog #aws

• 15:09 # Oof. Trying CloudFormation templates before I settle on terraform to make aws resources. Man oh man, the access policies. Yeesh. The trial and error and trial and error and trial and error, ad nauseum. It’s like programming assembly language in the 80s. There has to be a better way to make these templates. Maybe it’s terraform. #aws

Sunday 11/26

• 10:35 # Oh man it’s a deeeeeep rabbit hole when trying to setup infrastracture-as-code for an AWS Amplify project. CloudFormation templates, aws cli commands, the endless quirks and inconsistencies between aws services, the out-of-date or missing documentation, the sheer magnitude of time spent on permissions (probably 90% of time is spent on configuring access permissions), the endless questions for ChatGPT and/or Googling for answers because nothing makes intuitive sense, etc. etc. What an ordeal. One wonders if it’s this hellish in Azure. #aws #development