F# on The Personal Blog of Emily Gorcenski

Posts without posting: building a social media sharing relay with F♯ and Azure Functions, part 4

Sun, 08 Jan 2023 20:53:40 +0100

In the first three parts of the series, I set up an Azure Functions endpoint to accept a URL and a comment and share it across various social media sources. The last step of making this a proper workflow is to build the iOS shortcut.

The last two parts of this series were very long and tech heavy. This part is mercifully shorter and actually quite easy. The goal here is to create a shortcut in iOS to extract the URL and page title from a website I am visiting, prompt me for a comment, and then hit the API endpoint previously set up with an authentication key. This can be done with the built-in iOS Shortcuts app that ships with all current versions of iOS.

Posts without posting, a series

Architecting a relay

Writing the code

Configuring Azure Functions with Terraform

Setting up an iOS Shortcut

This is a very simple seven step process. In the iOS Shortcuts app, let’s create a shortcut called “Post Link”. The steps are as follows:

Get Any input from Share Sheet (Sharing -> System -> Share)
Ask for Text with Comment (Scripting -> Notification -> Ask for Input)
Set variable Tweet to Provided Input (Scripting -> Variables -> Set Variable)
Get Page URL from Shortcut Input (Web -> Safari -> Get Details of Safari Web Page)
Get Name from Shortcut Input (Web -> Safari -> Get Details of Safari Web Page)
Enter URL of endpoint (Web -> URLs -> URL)
Get Contents of URL (Web -> Web Requests -> Get Contents of URL)
Set Method to POST
Add a header with key set to x-functions-key and text set to your function key as found in the Azure Function you just set up

That’s it! Here are some images of how mine looks. To use this, whenever I am viewing a website, I can simply open the share tray and click “Post Link.”

Posts without posting: building a social media sharing relay with F♯ and Azure Functions, part 3

Sat, 31 Dec 2022 04:08:30 +0100

In the first two parts of the series, I introduced the architecture for building a link sharing relay and then built it with a little bit of F# code. This post covers setting up the infrastructure and deploying the code as an Azure Function.

Azure functions offer a really nice way to build serverless endpoints. Azure functions are phenomenally flexible, and in the previous part I showed how to set up an F# Azure function with an HttpTrigger. It runs locally, but to make it useful we need to deploy it to Azure. We’ll need to configure not a small amount of infrastructure to do so.

Posts without posting, a series

Architecting a relay

Writing the code

Configuring Azure Functions with Terraform

Setting up an iOS Shortcut

This post will be a little lengthy, but hopefully not as long as the last. I’ll use terraform to set things up, but there’s one step that doesn’t seem to be working. Configuring a Custom Domain for the Azure Function seems not to work, so this will be a small manual step. It’s not ideal, but it’s only one extra step for managing DNS. Some of my terraform scripts are inherited from previous work, but I’ll do my best to present them here as if I am using them for the first time.

To deploy the function, we’ll do the following:

Set up an Azure Resource Group
Set up Azure Storage for deploying the function code
Set up Azure Insights for monitoring
Set up a Key Vault for secrets management
Set up a Service Plan for the Azure Function
Set up an Azure Function
Configure a Key Vault Access Policy for the Azure Function
Deploy the function

Let’s get started!

Create an Azure Resource Group

First, we’ll set up a couple variables and data source in terraform.

# variables.tf

variable "domain" {
 type = string
 default = "emilygorcenski.com"
}


variable "regions" {
 type = map(string)
 default = {
 "primary" = "germanywestcentral"
 "functions-eu" = "northeurope"
 }
}

Not every region allows you to deploy Azure functions with a consumption plan (serverless), so I create some options in my variables. Next, we’ll set up our terraform provider. I’m ignoring the part where I have a remote terraform backend, but I recommend adding that, as well. My versions here are almost surely out of date, but I didn’t have time to fix that before writing this post. I’m on a schedule here!

# main.tf

terraform {
 required_providers {
 azurerm = {
 source = "hashicorp/azurerm"
 version = ">= 2.91"
 }
 azuread = {
 source = "hashicorp/azuread"
 version = "~> 2.0.0"
 }
 }
}

Finally, we can add the following to our main.tf file. Of course, variable names in ALL_CAPS should be replaced with something that makes sense for you.

resource "azurerm_resource_group" "rg" {
 name = "RESOURCE_GROUP_NAME"
 location = var.regions["primary"]
}

Setting up Azure storage

Once we have the resource group set up, we can configure a storage bucket that we can deploy our code into. Let’s create a new file, link-sharer.tf, and put the following block in it.

resource "azurerm_storage_account" "links_app_storage" {
 name = "STORAGE_NAME"
 resource_group_name = azurerm_resource_group.rg.name
 location = var.regions["functions-eu"]
 account_tier = "Standard"
 account_replication_type = "LRS"
}

No further action is needed here.

Configure Azure Insights

We’ll want to monitor our function, and while this step might be a little out of order, that’s ok. Let’s configure Azure Insights to let us monitor our function once we deploy. In link-sharer.tf, add:

resource "azurerm_application_insights" "links_afapp_insights" {
 name = "APP_INSIGHTS_NAME"
 location = var.regions["functions-eu"]
 resource_group_name = azurerm_resource_group.rg.name
 application_type = "web"
}

Set up a Key Vault

This step is a little more difficult. Let’s create a new file, key-vault.tf. We’ll start by configuring the Key Vault.

resource "azurerm_key_vault" "links_afapp_keyvault" {
 name = "KEYVAULT_NAME"
 location = azurerm_resource_group.rg.location
 resource_group_name = azurerm_resource_group.rg.name
 tenant_id = data.azurerm_client_config.current.tenant_id
 sku_name = "standard"
 soft_delete_retention_days = 7

 access_policy {
 tenant_id = data.azurerm_client_config.current.tenant_id
 object_id = data.external.account_info.result.object_id

 certificate_permissions = [
 "Create",
 "Delete",
 "DeleteIssuers",
 "Get",
 "GetIssuers",
 "Import",
 "List",
 "ListIssuers",
 "ManageContacts",
 "ManageIssuers",
 "Purge",
 "SetIssuers",
 "Update"
 ]

 key_permissions = [
 "Backup",
 "Create",
 "Decrypt",
 "Delete",
 "Encrypt",
 "Get",
 "Import",
 "List",
 "Purge",
 "Recover",
 "Restore",
 "Sign",
 "UnwrapKey",
 "Update",
 "Verify",
 "WrapKey"
 ]

 secret_permissions = [
 "Backup",
 "Delete",
 "Get",
 "List",
 "Purge",
 "Recover",
 "Restore",
 "Set"
 ]
 }
}

This will allow you to set up your Key Vault and gives your user account full permissions to manage keys. I think this is probably suboptimal, but I haven’t really dove into finding a better way to do this yet.

Next, in your main.tf file, put the following:

provider "azurerm" {
 features {
 key_vault {
 purge_soft_delete_on_destroy = true
 }
 }
}

This is needed because as of December 2020, all Key Vaults need to have “Soft Delete on Destroy” enabled.

Also in main.tf, let’s load our local data from our JSON for the secrets. I suppose we could also be reading these from environment variables, too, but this is how I do it.

locals {
 twitter_data = jsondecode(file("PATH/TO/TWITTER/CREDS/twitter-api.json"))
 mastodon_data = jsondecode(file("PATH/TO/MASTODON/CREDS/mastodon-api.json"))
 google_data = jsondecode(file("PATH/TO/GOOGLE/CREDS/google-key.json"))
}

Now, we can use this data in our key-vault.tf file to set up the secrets.

resource "azurerm_key_vault_secret" "google-sheets-key" {
 name = "GOOGLE-SERVICE-ACCOUNT-CREDENTIAL"
 value = base64encode(local.google_data.GOOGLE_SERVICE_ACCOUNT_CREDENTIAL)
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
}

resource "azurerm_key_vault_secret" "google-sheet-id" {
 name = "GOOGLE-SHEET-ID"
 value = local.google_data.GOOGLE_SHEET_ID
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
}

resource "azurerm_key_vault_secret" "twitter-consumer-key" {
 name = "twitter-consumer-key"
 value = local.twitter_data.TWITTER_CONSUMER_KEY
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
}

resource "azurerm_key_vault_secret" "twitter-consumer-key-secret" {
 name = "twitter-consumer-key-secret"
 value = local.twitter_data.TWITTER_CONSUMER_KEY_SECRET
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
}

resource "azurerm_key_vault_secret" "twitter-access-token" {
 name = "twitter-access-token"
 value = local.twitter_data.TWITTER_ACCESS_TOKEN
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
}

resource "azurerm_key_vault_secret" "twitter-access-token-secret" {
 name = "twitter-access-token-secret"
 value = local.twitter_data.TWITTER_ACCESS_TOKEN_SECRET
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
}

resource "azurerm_key_vault_secret" "mastodon-server" {
 name = "mastodon-server"
 value = local.mastodon_data.MASTODON_SERVER
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
}

resource "azurerm_key_vault_secret" "mastodon-access-token" {
 name = "mastodon-access-token"
 value = local.mastodon_data.MASTODON_ACCESS_TOKEN
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
}

That’s it for now. We’ll come back to this.

Set up a Service Plan

We need to set up a Service Plan where we specify our compute SKU and other details for the Azure function. This is easy. We want to use a Y1 SKU on a Windows OS. This is a consumption (serverless) SKU where you pay for what you use.

resource "azurerm_service_plan" "links_afapp_service_plan" {
 name = "SERVICE_PLAN_NAME"
 resource_group_name = azurerm_resource_group.rg.name
 location = var.regions["functions-eu"]
 sku_name = "Y1"
 os_type = "Windows"
}

Set up the Azure Function

With all this done, we can finally set up the Azure Function. In your main.tf, add the following:

resource "azurerm_windows_function_app" "links_app" {
 name = "AZURE_FUNCTION_NAME"
 resource_group_name = azurerm_resource_group.rg.name
 location = var.regions["functions-eu"]
 service_plan_id = azurerm_service_plan.links_afapp_service_plan.id
 storage_account_name = azurerm_storage_account.links_app_storage.name
 storage_account_access_key = azurerm_storage_account.links_app_storage.primary_access_key

 identity {
 type = "SystemAssigned"
 }

 site_config {
 use_32_bit_worker = false
 }

 app_settings = {
 "FUNCTIONS_WORKER_RUNTIME" = "dotnet",
 "AzureWebJobsDisableHomepage" = "true",
 "GOOGLE_SERVICE_ACCOUNT_CREDENTIAL" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.google-sheets-key.id})",
 "GOOGLE_SHEET_ID" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.google-sheet-id.id})",
 "APPINSIGHTS_INSTRUMENTATIONKEY" = azurerm_application_insights.il_afapp_insights.instrumentation_key,
 "TWITTER_CONSUMER_KEY" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.twitter-consumer-key.id})",
 "TWITTER_CONSUMER_KEY_SECRET" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.twitter-consumer-key-secret.id})",
 "TWITTER_ACCESS_TOKEN" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.twitter-access-token.id})",
 "TWITTER_ACCESS_TOKEN_SECRET" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.twitter-access-token-secret.id})",
 "MASTODON_SERVER" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.mastodon-server.id})",
 "MASTODON_ACCESS_TOKEN" = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.mastodon-access-token.id})",
 }
}

The app settings here are very important, this is how we load secrets in from the Azure Key Vault. This should automatically load in the secrets. However, in my experience, if I change a secret, it does not load the latest version. I’ll figure that out later.

Before we can make this work, however, we need to let the Function App read from the Key Vault.

Read secrets from the Key Vault

In your main.tf, add the following at the end.

resource "azurerm_key_vault_access_policy" "il-managed-id-policy-eu" {
 key_vault_id = azurerm_key_vault.links_afapp_keyvault.id
 tenant_id = azurerm_windows_function_app.links_app.identity.0.tenant_id
 object_id = azurerm_windows_function_app.links_app.identity.0.principal_id

 secret_permissions = [
 "Get"
 ]
}

Sometimes, on re-running terraform, this does not stick. I don’t know why. Sometimes I have to manually find the Principal ID, which is under “Identity” in the left nav in the Function App, and then copy the GUID into the “Access Policies” section within the Key Vault.

Set up DNS

I can’t terraform this, perhaps because of lack of supported functionality, but in the Function App you can configure a custom CNAME domain under “Custom Domains.” Simply copy the relevant information to your DNS provider.

Deploy the function

If the terraform works, then go ahead and apply the changes to your cloud, and let’s get ready to deploy the app! This is very simple. Navigate to your folder and type func azure functionapp publish AZURE_FUNCTION_NAME, replacing the last bit with your function name, of course.

You’ll want to go to “Configuration” in your Function App to see that secrets are being properly read. If so, great! Go ahead and send a cURL request and see how it works!

If it doesn’t, email me and I will try to help. I probably missed something or made a mistake, because what I present here is a bit modified from my own setup.

Thanks for following along. Next up, integrating with an iOS shortcut!

Posts without posting: building a social media sharing relay with F♯ and Azure Functions, part 2

Tue, 27 Dec 2022 18:52:54 +0100

In the first part of the series, I introduced the architecture for building a link sharing relay. This post covers the actual code needed to make that relay work. I’m writing in F#, a beautiful language.

Part 2: Writing the code

Azure functions provide a simple environment to deploy code, requiring a minimum of development overhead. Nevertheless, Azure Functions are not low-code or no-code solutions. We will need to write some code. For this project, I chose F#, so that I could develop my skills in the language and learn a bit more about functional programming. I also wanted to try out the Railway Pattern, which absolutely blew my mind when I first read about it. It’s an awesome way to write code, and I can’t say I’ve mastered it, but I wanted to try it out.

Posts without posting, a series

Architecting a relay

Writing the code

Configuring Azure Functions with Terraform

Setting up an iOS Shortcut

To get started, you’re going to need the .Net 6.0 runtime and the Azure Functions Core Tools. I strongly recommend using VSCode with the Ionide extension. I’m developing on Ubuntu 20.04 LTS on Windows System Linux (WSL) 2, but you should be able to do this on Mac, Linux, or Windows, if you prefer.

This will be a long post, so I’ll try to break it down for you. We’ll do the following steps.

Set up an Azure Function and familiarize with some key files and configurations
Build the run function and configure it to accept POST requests
Introduce the railway pattern and how to adapt it for async programming
Integrate external services
Test locally

Step 1: Set up an Azure Function

Let’s begin at the beginning. We’re going to set up an azure function. You can do this wherever you’d like, but I did it in my static site repository. This might have been a mistake, so I’ll recommend you do this in its own folder. You won’t have to create it, the tools should set everything up for you. Assuming you’ve installed the Azure Functions Core Tools linked above, simply type

func init [PROJECT NAME]

Replace [PROJECT NAME] with your project’s name. This should prompt you to select a worker runtime:

Select a number for worker runtime:
1. dotnet
2. dotnet (isolated process)
3. node
4. python
5. powershell
6. custom
Choose option:

Choose Option 1, for dotnet. Option 2 might resolve the issue I was having with .Net 7.0, but I haven’t tested this yet, so let’s stick with what’s known and easy. You’ll next be prompted to pick a language. Choose F#, if you dare.

This should create a project folder for you, initialize it as a git repository, create a .gitignore, and create a handful of files: [PROJECT NAME].csproj, host.json, local.settings.json, and a couple of directories.

Navigate to this file, and let’s initialize a new function. Type func new -n [FUNCTION NAME] -l F# to create a new function [FUNCTION NAME]. This can be the same as your project name, but need not be. This is because a single function app deployment can have multiple endpoints, each pointing to a different function. When prompted, choose HttpTrigger. Running this command should add a couple of new files: [FUNCTION NAME].fs, metadata.json, and function.json.

Let’s go ahead and rename [PROJECT NAME].csproj to [PROJECT NAME].fsproj. I don’t know that it matters, but I feel a little weird calling an F# project like a C# project.

Let’s open the .fsproj file, which should have some XML inside. Find the block that says

<ItemGroup>
 <None Update="host.json">
 <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
 </None>
 <None Update="local.settings.json">
 <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
 <CopyToPublishDirectory>Never</CopyToPublishDirectory>
 </None>
</ItemGroup>

We need to do some small changes here. Update the block as follows:

<ItemGroup>
 <Compile Include="[FUNCTION NAME].fs" />
 <None Include="host.json">
 <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
 </None>
 <None Include="local.settings.json" Condition="Exists('local.settings.json')">
 <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
 <CopyToPublishDirectory>Never</CopyToPublishDirectory>
 </None>
</ItemGroup>

The first change ensures that we compile our F# code and include it in the build. The second changes involve modifying the Update attribute to be Include. This is necessary for local testing. Finally, we add a Condition attribute to further simplify local testing.

Open your .fsproj file, and in the second <ItemGroup> block, just before <None Include="host.json"> add the following snippet: <Compile Include=[FUNCTION NAME].fs" />. This ensures that your code is compiled and built. You’ll need to do this for every file that you add.

In VSCode (or your preferred IDE), open [FUNCTION NAME].fs. It should have an autogenerated template that looks something like this. From now on, I’ll be using LinkSharer as my project name and function name.

namespace LinkSharer

open System
open System.IO
open Microsoft.AspNetCore.Mvc
open Microsoft.Azure.WebJobs
open Microsoft.Azure.WebJobs.Extensions.Http
open Microsoft.AspNetCore.Http
open Newtonsoft.Json
open Microsoft.Extensions.Logging

module LinkSharer =
 // Define a nullable container to deserialize into.
 [<AllowNullLiteral>]
 type NameContainer() =
 member val Name = "" with get, set

 // For convenience, it's better to have a central place for the literal.
 [<Literal>]
 let Name = "name"

 [<FunctionName("LinkSharer")>]
 let run ([<HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)>]req: HttpRequest) (log: ILogger) =
 async {
 log.LogInformation("F# HTTP trigger function processed a request.")

 let nameOpt =
 if req.Query.ContainsKey(Name) then
 Some(req.Query.[Name].[0])
 else
 None

 use stream = new StreamReader(req.Body)
 let! reqBody = stream.ReadToEndAsync() |> Async.AwaitTask

 let data = JsonConvert.DeserializeObject<NameContainer>(reqBody)

 let name =
 match nameOpt with
 | Some n -> n
 | None ->
 match data with
 | null -> ""
 | nc -> nc.Name

 let responseMessage =
 if (String.IsNullOrWhiteSpace(name)) then
 "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response."
 else
 "Hello, " + name + ". This HTTP triggered function executed successfully."

 return OkObjectResult(responseMessage) :> IActionResult
 } |> Async.StartAsTask

This is boilerplate code and it should run locally. Fun func host start and try sending a request with curl to the URL that it displays, typically something like curl http://localhost:7071/api/FUNCTION NAME. You should see a message like, This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response. If so, congratulations! We’ve configured our F# function, let’s move onto the next step.

Step 2: Build the `run` function

To build our link sharer, we’re going to strip out a lot of the boilerplate and cruft. This is actually going to be very simple. We’ll start with the run function. In your LinkSharer.fs file, let’s modify it as follows.

[<FunctionName("LinkSharer")>]
let run ([<HttpTrigger(AuthorizationLevel.Function, "post", Route = null)>]req: HttpRequest) (log: ILogger) =
 async {
 log.LogInformation("F# HTTP trigger function processed a request.")
 let! (data: LinkData) = getPostFromReq req

 let tweet: Async<Result<ID,string>> =
 getTwitterClient
 >>@ writeTweet data

 let sheet: Async<Result<ID,string>> =
 getSheetService
 >>@ writeToGoogleSheet data

 let toot: Async<Result<ID,string>> =
 getMastodonClient
 >>@ writeToot data

 let! (result: Result<ID,string>) =
 tweet
 >>== toot
 >>== sheet

 return match result with
 | Success s -> OkObjectResult(s) :> IActionResult
 | Failure f -> StatusCodeResult(500) :> IActionResult

 } |> Async.StartAsTask

This is actually shockingly simple. First, I’ve stripped off get from the function call, as this will be a POST-only endpoint. Notice AuthorizationLevel.Function, this forces us to authenticate to use the API. We don’t want randos sharing to our social media!

Next, we have let! (data: LinkData) = getPostFromReq req. This takes the JSON payload attached to the request and processes it with a function called getPostFromReq. Essentially, all this does is take the request data and format it using an F# Record type. Let’s create that code now. Above the run method, add the following code.

type LinkData = {
 url: string
 title: string
 comment: string
}

let getPostFromReq (req: HttpRequest) =
 async {
 use stream: StreamReader = new StreamReader(req.Body)
 let! (reqBody: string) = stream.ReadToEndAsync() |> Async.AwaitTask
 return JsonConvert.DeserializeObject<LinkData>(reqBody)
 }

We probably don’t need the async here, but let’s use it. What this does is reads the JSON data in the payload and deserializes it to a LinkData type that we can use easily elsewhere in the code. This is the easy part. Let’s look now to the end of the function, where things start to get interesting.

Step 3: Introducing the railway pattern

Let’s skip over some code and look at the final block, because this is where things get interesting.

let! (result: Result<ID,string>) =
 tweet
 >>== toot
 >>== sheet

return match result with
 | Success s -> OkObjectResult(s) :> IActionResult
 | Failure f -> StatusCodeResult(500) :> IActionResult

The >>== operator isn’t standard F# syntax. I had to define this operation. This kind of comes from the Railway Oriented Programming blog post, which introduces &&& as a parallel operator. I won’t explain this all here, I encourage you to read the post and absorb it. It’s brilliant. However, the methods presented in the blog post don’t work as-is for async programming. I had to modify them. Using a Code Review Stackexchange post as inspiration, I defined the >>== operator to do async parallel railway switches, because with the Fira Code ligatures I use, it looks nicely parallel.

To make this work at all, I have to introduce a couple helper functions and types.


type Result<'TSuccess,'TFailure> =
 | Success of 'TSuccess
 | Failure of 'TFailure
type ID = { id : string }

let bindAsync<'t,'s,'terr> (binder:'t -> Async<Result<'s,'terr>>) (result:Async<Result<'t,'terr>>) : Async<Result<'s,'terr>> =
 async {
 let! res = result
 match res with
 | Success s -> return! binder s
 | Failure f -> return Failure f
 }

let plusAsync addSuccess addFailure (switch1 : Async<Result<'s,'terr>>) (switch2 : Async<Result<'s,'terr>>) =
 async {
 let! res1 = switch1
 let! res2 = switch2
 return match (res1),(res2) with
 | Success s1,Success s2 -> Success (addSuccess s1 s2)
 | Failure f1,Success _ -> Failure f1
 | Success _ ,Failure f2 -> Failure f2
 | Failure f1,Failure f2 -> Failure (addFailure f1 f2)
 }

let (>>@) twoTrackInput switchFunction =
 bindAsync switchFunction twoTrackInput

let (>>==) v1 v2 =
 let addSuccess r1 r2 = {id=r1.id + "; " + r2.id}
 let addFailure s1 s2 = s1 + "; " + s2
 plusAsync addSuccess addFailure v1 v2

The key here is the introduction of the Result<'TSuccess, 'TFailure> type and the terribly named ID record type, which will capture relevant information for successful posts; namely, the post ID.

The >>@ function is an asynchronous (hence the @) version of the >>= bind-with-piping operator, which calls the bindAsync function which, as you can guess, is the async equivalent of the bind operator from the original blog post. The plusAsync function is the asynchronous version of the plus function from the original blog post. All in all, if you can grok the Railway Oriented Programming post, these modifications are not that much more complex. They’re simply adapted for the Async<Result<'s,'terr>> return type.

When we get to the end of the code, we have:

return match result with
 | Success s -> OkObjectResult(s) :> IActionResult
 | Failure f -> StatusCodeResult(500) :> IActionResult

This is a really nice way to either return a 200 OK status code, along with a message that includes our successful IDs, or a 500 Server Error code, which, unfortunately, I haven’t figured out how to better manage.

If you’ve made it this far, then the remaining code in this function should be very easy to understand.

let tweet: Async<Result<ID,string>> =
 getTwitterClient
 >>@ writeTweet data

let sheet: Async<Result<ID,string>> =
 getSheetService
 >>@ writeToGoogleSheet data

let toot: Async<Result<ID,string>> =
 getMastodonClient
 >>@ writeToot data

All that this does is define functions that use the >>@ async bind-with-pipe operator to post the link to the various services. I really like this code. The run method is very clean, and aside from the subtleties of the syntax, the structure of the code makes it very clear what is happening. A little bit of work makes the code much, much cleaner. All we need to do now is write the code for each of these services. Ready for that? Let’s move onto the next part.

Step 4: Integrate external services

For my link sharing function, I want to push content to (at least) three different sinks: a Google Sheets, which serves as kind of a bootleg database; Twitter; and Mastodon. To do this, I’ll use the Google Sheets API, Tweetinvi, and Mastonet. These are all fairly easy to install with the dotnet package manager.

One of the beautiful things about F# is that you can use any .Net library with it, even if it wasn’t built for F#. That means any C# library works with F# equally well. This is a long post and I’m not going to belabor how to do all of these things, but rather, I’ll cover the key steps and present the code.

Google Sheets

This is by far the most complex integration because to make this work you need to use Google Cloud. Start by creating a new Google Sheet (you can just go to sheets.new), or if you don’t have a Gmail account, create one first and then create a new Google Sheet. You can name it whatever, I called mine “Interesting Links”. In the first row, name the columns as follows: url, title, added, comment.

Next, navigate to the Google Cloud console. Create a new project, and call it something obvious, like Interesting Links. Select the project and navigate to the project dashboard. In the hamburger menu in the top left, go to “APIs & Services” and click “Enabled APIs & Services”. There, at the top of the screen, click “+ Enable APIs and Services”. In the search bar, click “Google Sheets” and enable the API.

Once done, click “Manage”. On the left, you should see “Credentials”. Click this and you should see “API Keys”, “OAuth 2.0 Client IDs”, and “Service Accounts”. We’ll use a Service Account, although this may not be the best option. Perhaps in a future update I will fix this. Click “Create Credentials” at the top and choose “Service Account”.

Give the Service Account a meaningful name, and click “Create and Continue.” On the next screen, it will ask you to choose a role. In the search box, type “Service Account Token Creator”. Skip the next step and click “Done”. This will return you to the Credentials screen. Click the Service Account you created, and at the top of the screen, click “Keys.” Next, click “Add Key” and “Create New Key”. This should pop a dialog box. Select “JSON” and click “Create”. This will create a JSON file for you and download it. Copy it somewhere meaningful, such as ~/.config/gcloud.

We’ll base64 this string, and the easiest way with the command line is simply base64 -w 0 ~/.config/gcloud/[KEY FILE].json.

For local testing, we can either create an environment variable with this file, or we can put it in local.settings.json with a field named GOOGLE_SERVICE_ACCOUNT_CREDENTIAL. While we’re doing this, go to the spreadsheet you created and copy the sheet ID. It’s the long alphanumeric string that should be between d/ and /edit. Put this in a variable called GOOGLE_SPREADSHEET_ID.

Lastly, copy the Service Account email you created, which can be found on the Credentials page we visited earlier. In your Google Sheet, in the top right click “Share” and paste this email, giving it Editor permissions.

All of this could have probably been done with terraform, but I’m going to be real. This is a one-time setup, and it’s complicated. I might have gotten it wrong here, but you can also Google how to set up a Google Sheets API credential. I don’t think it’s worth figuring out the terraform. Alternatively, we could have used Workload Identity Federation, but this, too, is overkill.

Once we have the account set up, we can write the code to write to a spreadsheet. It’s fairly straightforward. First, we have to build a SheetsService using the Google Cloud SDK. This requires our credential. Here, I define getSheetService which does all of the work to turn the environment variable into a credential and uses this to initialize a SheetsService. Here, we’re returning a Success type containing the object we just initialized, so we need to create a record type, Sheet, for that as well.

open Google.Apis.Auth.OAuth2
open Google.Apis.Sheets.v4
open Google.Apis.Sheets.v4.Data
open Google.Apis.Services

// ...

type Sheet = { service : SheetsService }

// ...

let getSheetService =
 async {
 let scopes: string list = [ SheetsService.Scope.Spreadsheets ]

 let getServiceAccountCredential (googleCredential: GoogleCredential) =
 googleCredential.CreateScoped(scopes).UnderlyingCredential
 :?> ServiceAccountCredential

 try
 let credential : ServiceAccountCredential =
 System.Environment.GetEnvironmentVariable("GOOGLE_SERVICE_ACCOUNT_CREDENTIAL")
 |> Convert.FromBase64String
 |> Text.Encoding.UTF8.GetString
 |> GoogleCredential.FromJson
 |> getServiceAccountCredential

 let initializer: BaseClientService.Initializer =
 new BaseClientService.Initializer(HttpClientInitializer=credential,
 ApplicationName=ApplicationName)

 return Success {service=new SheetsService(initializer)}
 with
 | ex -> return Failure ex.Message
 }

let writeToGoogleSheet (data: LinkData) (input : Sheet) =
 async {
 let service: SheetsService = input.service
 let sheetId: string = System.Environment.GetEnvironmentVariable("GOOGLE_SHEET_ID")
 let range: string = "A:D"

 let newItem: List<IList<Object>> = new List<IList<Object>>();
 let obj: List<Object> = new List<Object>([|data.url :> Object;
 data.title;
 DateTime.UtcNow;
 data.comment|]);
 newItem.Add(obj)

 let request =
 service.Spreadsheets.Values.Append(new ValueRange(Values=newItem),
 sheetId,
 range,
 InsertDataOption=SpreadsheetsResource.ValuesResource.AppendRequest.InsertDataOptionEnum.INSERTROWS,
 ValueInputOption=SpreadsheetsResource.ValuesResource.AppendRequest.ValueInputOptionEnum.USERENTERED)
 try
 let! response = request.ExecuteAsync() |> Async.AwaitTask
 return Success {id=response.Updates.UpdatedRange}
 with
 | (ex: exn) -> return Failure ex.Message
 }

In writeGoogleSheets, we consider each row to be a list of objects, and each sheet range to be a list of rows, and hence a list of lists of objects. I know I have four columns, so I can simply define my sheet range using open-ended columns, or A:D. We create a new row with the relevant fields, url, title, added, and comment, although they appear here unnamed, and simply append this using the API. We construct the request, and by using try-with, we can either return a Success if the operation succeeds, or a Failure containing an exception message if the operation fails.

This code is again very simple. We’re simply fetching a credential, initializing a sheet service, crafting the data to be added, and adding it.

Twitter

Google Sheets was very complicated, but Twitter is a bit easier. To post to Twitter, the important thing to know is you’ll need an Individual Developer Account with elevated privileges. This can be done at the Twitter Developer Portal, but I’ll be honest the portal is badly designed and things are really hard to find. It took me an hour, and now that I’ve done it once, I can’t go back to figure out how to tell you how to do it. All I can tell you are the important words: individual developer account and elevated privileges. Click around. Godspeed.

You’ll need to create a new project, I called mine “Emily’s Syndicator”. This allows me to post statuses to Twitter. Once you do this, on the left you should see your project. Click it, and toward the top of the screen you should see “Keys and tokens.” Here, you can generate the required keys. I’m using an “Access token and secret”, which I copy to a JSON file locally and also can put in local.settings.json in fields called TWITTER_ACCESS_TOKEN and TWITTER_ACCESS_TOKEN_SECRET for local testing.

Next, generate an API key and secret, sometimes called a consumer key and secret, and store these locally in fields named TWITTER_CONSUMER_KEY and TWITTER_CONSUMER_KEY_SECRET.

There are better ways to authenticate with Twitter, but everything is badly documented. This works, and I’ll improve it later. To post, I’m using the Tweetinvi SDK. Tweetinvi hasn’t been updated in a couple of years, but it works. Just like with Google Sheets, we’ll need to create a record type to receive the Success case, otherwise the workflow is even simpler.

open Tweetinvi

// ...

type Twitter = { twitterClient : TwitterClient }

// ...

let buildPostFromData (data: LinkData) =
 data.comment + "\n\n" + data.url

// ...

let getTwitterClient =
 async {
 try
 return Success {twitterClient=new TwitterClient(System.Environment.GetEnvironmentVariable("TWITTER_CONSUMER_KEY"),
 System.Environment.GetEnvironmentVariable("TWITTER_CONSUMER_KEY_SECRET"),
 System.Environment.GetEnvironmentVariable("TWITTER_ACCESS_TOKEN"),
 System.Environment.GetEnvironmentVariable("TWITTER_ACCESS_TOKEN_SECRET"))}
 with
 | ex -> return Failure ex.Message
 }

let writeTweet (data: LinkData) (input : Twitter) =
 async {
 let client = input.twitterClient
 let tweet = buildPostFromData data
 try
 let post = client.Tweets.PublishTweetAsync(tweet)
 return Success {id=post.Result.Id.ToString()}
 with
 | ex -> return Failure ex.Message
 }

This code is extremely straightforward. First, we construct the client using the Twitter credentials obtained as described above. Then, we construct a post from the comment and url fields in the JSON payload. Last, we publish the Tweet and return the Tweet ID. It really could not be simpler.

Mastodon

Mastodon is even easier than Twitter. The setup is quite simple. From your Mastodon account on web, click on “Preferences” and then choose “Development”. Create a new application, give it a meaningful name, and unselect all scopes except write:statuses. Click “Submit” and you’ll retrieve an Authentication token. Store this in a local environment variable (or in local.settings.json, as before) called MASTODON_ACCESS_TOKEN and while we’re here, put your server name in as well, as MASTODON_SERVER.

Once complete, we’ll follow nearly the identical steps we did for Twitter, just using the Mastonet SDK instead.

open Mastonet

// ...

type Mastodon = { mastodonClient: MastodonClient }

// ...

let getMastodonClient =
 async {
 try
 let client = new MastodonClient(System.Environment.GetEnvironmentVariable("MASTODON_SERVER"),
 System.Environment.GetEnvironmentVariable("MASTODON_ACCESS_TOKEN"))
 return Success { mastodonClient=client }
 with
 | ex -> return Failure ex.Message
 }

let writeToot (data: LinkData) (input: Mastodon) =
 async {
 let client = input.mastodonClient
 let post = buildPostFromData data
 try
 let result = client.PublishStatus(post).Result
 return Success { id=result.Id }
 with
 | ex -> return Failure ex.Message
 }

Piece of cake.

With each of these, notice how I split the process up into two steps: first we construct the client or service, and then we execute the posting. These are all asynchronous operations, and we chain them together using the >>@ async bind-with-piping operator. We bind each of those, which lets us then parallelize them as described earlier. Altogether, the code should be very easy and very minimal. The hard part is configuring the various accounts.

Once we put this all together, it should come out to ~200 lines of code. I’ve copied my file (which might have some small differences) to a public Gist.

Testing locally

If everything is working right, then you should be able to execute this locally fairly easily. As before, run func host start, and it should fire up a webserver at localhost:7071. Use a cURL request to post a JSON payload as follows:

curl -X POST http://localhost:7071/api/LinkSharer -d '{"url": "test", "title": "test title", "comment": "test comment"}' -v

If everything works, go check your social media services! You should see test messages. Don’t forget to delete these messages, of course ;)

Wrapping up

This was a very long post, but hopefully you followed it. I tried to structure it step-by-step. Of course, you can skip any of the social media services. If you don’t want to use Google Sheets as a data store, then you can just ignore all that! Altogether this isn’t a lot of code, and I like how the code can be very structured and clean. There’s very little boilerplate, and you can really focus on the functional flow of the code, rather than state management and object handling. The railway pattern works really well with asynchronous programming. I like this result, even if the setup was a pain in the ass.

We’re still not out of the woods yet. We still need to set up Azure to deploy this function to a live environment. I’ll write that post tomorrow.

For now, I hope you liked this F#-based approach, and thanks for sticking it out with this long post!

Posts without posting: building a social media sharing relay with F♯ and Azure Functions, part 1

Tue, 27 Dec 2022 04:20:56 +0100

Social media is getting more complicated, and building tools to automate sharing content can help simplify the fragmentation of social networks. Here’s how I post without having to log into social media.

Part 1: Architecting a relay

Sharing content across multiple social media feeds can be a real pain. Social media links are fragile. Networks can ban you, or force you to remove your posts. They often use link shorteners, modifying the original link, which adds another possible point of failure. Social media is built for the moment, and it’s not built to make bridging networks easy. If you want to share in multiple places, or to keep a record of what you’re sharing, including the original, unmodified URL, that can be a challenge.

Posts without posting, a series

Architecting a relay

Writing the code

Configuring Azure Functions with Terraform

Setting up an iOS Shortcut

After Elon Musk’s takeover of Twitter, I decided to stop using the site day-to-day. Instead, I want to use it, and my relatively large following there, to act as a syndication engine, following the Indieweb principles of POSSE: Publish Own Site, Syndicate Elsewhere. I still want to share what I find interesting, and I want to share what I write and create. But as I migrate to the indieweb.social server on Mastodon, I want my sharing to be automated and mindless. My ideal workflow is to read an interesting article, press a button, type a message, and automatically share the link to the article with my comment everywhere I have an online presence, all without requiring any third-party apps.

For me currently, that means sharing in three places: Twitter, Mastodon, and the front page of this site. Before building this solution, I was using Zapier to serve as a relay for this content. However, Zapier’s free tier does not let you use webhooks, and the service does not have a Mastodon plug-in. This mean that to use REST functionality, including cross-posting to Mastodon, I had to subscribe to their cheapest tier, which at around $20 a month is not cheap. I wanted a cheaper solution, one I could manage entirely myself. Technically, I could have done all of this entirely with an iPhone Shortcut, if I had the Twitter app installed. Since I haven’t had the app on my phone in almost a year, this wouldn’t do. As I’ll explain in this post, I’ll still use an iPhone Shortcut, but I’m pushing most of the business logic to a serverless Azure Function App. This is a solution that is practically free, if you know how to do it. Lucky you, over the next few posts, I’m going to show you how.

In this post, I’m going to describe the architecture of how I set this up. I do not know how to build something similar with Android; I do not have access to an Android phone at the moment. I’m almost positive, however, that it can be done with similar ease.

To get started, here’s the tech I’m going to use.

Microsoft Azure — For low-cost hosting, and because I use it to host the rest of my site
Terraform — For setting up the Azure Functions App using Infrastructure-as-Code
F# — Because I love the language and I want to practice it more
VSCode — A fantastic development environment
iOS Shortcuts — A built-in app on my iPhone to let me build the workflow for free
Google Sheets — A simple, free, high-performance data store for efficient reads, infrequent writes, with an unbeatable SLA for the cost
Github Actions — A simple, low-cost CI/CD environment

To get started, you’ll need:

A Github account (free)
An Azure account (costs vary and you need a credit card to set this up, but I pay ~$9 a month for almost all my hosting needs. The feature described in this post costs me $0.01 monthly)
A Gmail account (free)
A Google cloud account attached to that Gmail account (free, you might need a credit card here as well, but what I am discussing costs exactly $0 monthly)

The link sharing workflow begins when I read an interesting article on my phone. Using the default iOS share tray, I add a sharing option that collects the page URL, the page title, and opens a dialog for me to write a comment about the post. I’ll call this shortcut Post Link. After writing a comment, I click Submit, and the shortcut packages everything into a small JSON payload and calls a serverless Azure Function App REST API authenticated with an authentication token, which gets passed along in a header.

The Function App, which I call LinkSharer, takes the JSON payload, crafts a simple post, and performs a sequence of asynchronous API calls to put the link and the comment to various social media outlets. For my website, I also write another Function App called LinkFetcher which collects the last n links and displays them on my website.

Altogether, the workflow looks something like this.

The first question you might ask is, “Google Sheets? Really?” My answer is yes, really. For mostly-read, seldom-write, rarely-update workflows, Google Sheets is probably the lowest friction, lowest cost option you have for a durable backing store out there. Simply put, you’re not going to beat Google’s SLA with a free offering. You don’t have to worry about backups, it’s dirt-simple to set up, hand-editable, and it is highly-performant. Heck, it even comes with an out-of-the-box JSON API if you don’t want to write your own reader facade. Google Sheets has limitations, big ones. But for quick-and-dirty JAM stack development, there are worse options out there. Eventually, I will replace this with a proper database. But I’m not ready to spend the money yet and I want to focus on the rest of the feature work. So Google Sheets it is.

In terms of my choice of Azure, I’ve been on Azure for a couple of years now. It has limitations, but for personal use, it’s great. I spend $5-10 monthly for everything. That will increase when I set up a proper database, but most of the cost right now is the CDN. Essentially, for $5 monthly I can host my site and have incredible DDoS protection. It’s not bad. But my favorite feature of Azure is that I can deploy Azure Functions written in .Net, which means I can write F#. If you haven’t tried F#, it’s an amazing language. I’m not a very good functional programmer; if you ask me what a monad is, I’m going to lecture you how real mathematicians think Category Theory is a bunch of abstract useless gibberish, and I’m also not going to answer your question because I don’t actually know. I like F# because the community is averse to this kind of jargon, but also because the language is beautiful and fun to write. Honestly, I wish I could write it every day.

Enough asides. To get started on the above workflow, I’ll develop in three phases.

First, I’ll actually code up the Azure Function and run it locally. This will let me test everything before deployment. I’ll need the Azure Functions Core Tools and the .Net runtime installed for this. (As of this writing, I am using .Net 6.0, as .Net 7.0 still has some kinks when it comes to Azure Functions.)

In the second step, I’ll set up my infrastructure so that I can actually deploy the code to Azure Functions. This will involve setting up the function, and also setting up some Azure Key Vault secrets. There are a couple bits I haven’t figured out how to do, so it’s not pure infra-as-code here. Namely, it seems like Azure Function App Custom Domains aren’t working with Terraform as of this writing. In this step I’ll also set up a CI/CD workflow with Github actions to automate the deployment of the Azure Function.

Third, I’ll shift my focus to a low-code environment, and build a simple iOS shortcut to use the workflow.

The writeups for these posts I’ll be publishing in the following days. In the meanwhile, this post should give you an overview of the workflow and the architecture. It’s not complex, but some of what I describe might feel overly technical. If you want a simplified version of these posts at some point, let me know. I’m confident that the average person familiar with a computer can get this working for themselves with some help.

In the meantime, I’m really enjoying being able to share content with all of my followers without having to log into social media.

Until tomorrow.

F# on The Personal Blog of Emily Gorcenski

Posts without posting: building a social media sharing relay with F♯ and Azure Functions, part 4

Posts without posting, a series

Posts without posting: building a social media sharing relay with F♯ and Azure Functions, part 3

Posts without posting, a series

Create an Azure Resource Group

Setting up Azure storage

Configure Azure Insights

Set up a Key Vault

Set up a Service Plan

Set up the Azure Function

Read secrets from the Key Vault

Set up DNS

Deploy the function

Posts without posting: building a social media sharing relay with F♯ and Azure Functions, part 2

Part 2: Writing the code

Posts without posting, a series

Step 1: Set up an Azure Function

Step 2: Build the run function

Step 3: Introducing the railway pattern

Step 4: Integrate external services

Google Sheets

Twitter

Mastodon

Testing locally

Wrapping up

Posts without posting: building a social media sharing relay with F♯ and Azure Functions, part 1

Part 1: Architecting a relay

Posts without posting, a series

Step 2: Build the `run` function