Blink (ESP32)

If you are new to the Arduino/ESP32 world, blink is the classic “Hello World!” program for the Arduino developer. This exercise involves using the build-in LED on pin 13 of many Arduino boards or wiring to an LED to an one of the many digital pins with a 220 ohm ballast resistor to limit the current flow. Since the ESP development board does not have a built-in LED I used the following wiring example.

I my case, I used the breakout board and an Ideal Lever Wire Connector to connect the LED and 220 ohm resistor without the need to solder.

The blink sketch is included in this project in the folder labeled /Blink. With the ESP32 Development Board I chose there is no built in LED so I had to manually redefine the LED_BUILTIN constant in the code. In my case I used pin 4. If you are new to programming an Arduino/ESP32 I recommend starting by reading “Getting Started with Arduino IDE 2”. You can also search the internet for “Arduino blink” or “programming the Arduino” to find a great deal of articles and tutorials.

Beyond Blink (ESP32)

Beyond Blink is an exercise to look at the simplicity and power the ESP32 has when it comes to connecting to your Wi-Fi network and the Internet. The code was adapted from the Last Minute Engineers - Learn Electronics the Easy Way article In-depth: Create A Simple ESP32 Web Server In Arduino IDE (lastminuteengineers.com) and demonstrated how to connect your ESP32 to a Wi-Fi network hosing a web page that presents a button that can be used to turn the LED on and off.

This exercise uses the same wired LED from the Blink exercise. You will again need to edit the code to modify LED_BUILTIN if you are using a pin other than 4 for the LED. You will also need to edit the configuration.h header file to add your Wi-Fi SSID and password. The code can be found in the /BeyondBlink folder.

Is the Power On (220V Optocoupler Module)

As part of this project I need to be able to see if the power is on. In particular, I want to monitor the pump timer, the air compressor pump timer, the high water alarm and the air compressor alarm.

To do this I will use a simple 220V optocoupler module (pictured above) which can be connected to the hot 120V lead (‘L’ for load, typically a black wire) of any of the items I previously listed and common (‘N’ for neutral, typically a white wire) on one side of the module. On the other side I connect the module to 3.3 voltage (VCC - positive voltage), ground (GND - ground, negative) and one of the GPOI (General-Purpose Input-Output) pins of the ESP32 (OUT - signal). You can learn more about the ESP32 GPOI pins at ESP32 Pinout Reference: Which GPIO pins should you use? - Random Nerd Tutorials. Below is a diagram for a test circuit.

The following is a photo of the AC circuit test board I built for testing based on the test circuit above. When plugged in and the the switch is on, the plug has power and the optocoupler module (green PCB (Printed Circuit Board), far right of photo) will sense voltage. In the next module I will discuss the current sensor (blue PCB, bottom of photo).

The code can be found in the /IsThePowerOn folder and is extremely simplistic. Basically when power is sensed the LED I wired in the previous exercises is turned on and a message is sent to the serial port. As before you will need to change the LED_BUILTIN constant if you use a pin other than GPIO 4. You will also need to change the POWER_ON_OPTOCOUPLER_MODULE constant if you use a GPIO pin other than 34. In the photo below VCC on the optocoupler is wired to 3V3, OUT to GPIO pin 34 and GND to GND on the development breakout board.

Is the Pump Running (Current Sensor)

While determining if the pump has been turned on by either the pump timer or the high water override float is a simple mater of using a 220V optocoupler module to detect voltage, I can’t always be certain that the pump is running since a low water float is tied directly to the pump circuit after the control box wiring. The low water float is wired in to the circuit to keep the pump from running and possibly burning out when there is no water in the septic clear water tank.

Now I could loop back a wire from the low water float/pump connection and wire it to a 220V optocoupler module to determine if the pump was running, but depending on the distance of the control box from the pump this could be extremely difficult. Also, since the low water float is most likely wired directly to one of the pump leads, it is also most likely connected with a special water proof heat shrink connector which complicates things, and making this project more complicated is not an objective.

Using a current sensor that can be easily clipped around the power lead leaving the control box is much easier. For this reason I chose the Gravity 20A Analog AC Current Sensor from DFRobot. I did originally look at using a cheaper current sensor based on the ZMCT103C, but after several hours of trying to get a reading, the fact that it had a 5A maximum, requires 5V to power and did not easily clip on to a wire I moved on to this sensor.

The DFRobot current sensor is easy to clip on to a wire, connect to the ESP32 and the example code was easy to implement with only one hitch discovering that the ESP32 12 bit analog requires you divide by 4096 verses 1024. In the photo below the sensor is connect with the positive lead (+) connected to the ESP32 3.3V (3V3), the negative lead (-) to ground (GND), the signal (A) connected to GPIO pin 35 and the AC transformer probe clipped on to one power lead.

The code can be found in the /IsThePumpRunning folder and is based off of the vendors example code. The LED will blink to indicate activity and the amperage will be displayed on the serial port about once every second. As before you will need to change the LED_BUILTIN constant if you use a pin other than GPIO 4. You will also need to change the AC_CURRENT_SENSOR constant if you us a GPIO pin other than 35.

Where is my Data (SD Card Reader)

Later in this project I will look at logging data. To do this I will use an SD card. SD cards can store large amounts of data in a file format that can be read by any personal computer. The SD card reader I chose to use is operated on 3.3V and you should note the description on Amazon is incorrect about it operating on 5V. You will find other SD card reader modules that work on 5V which are compatible with 3.3V inputs. I simple chose the 3.3V version to stock my workbench with modules I could use with future ESP32 projects that might run off of batteries.

The SD card reader module communicates using SPI communication protocol. You can connect it to the ESP32 using the default SPI pins.

To test the module you will first want to format the micro SD card using your personal computer. Most laptops have an SD card slot you can use with an micro SD card adaptor or you may need to purchase a USB SD card reader. On a Windows machine you will open file manager, select the card and format it as FAT32.

The test code can be found in the /WhereIsMyData folder and is pulled from the ESP32 SD example code SD_Test with the SD_CS set to GPIO pin 5. The following are two good tutorials on using SD card readers with the ESP32.

ESP32: Guide for MicroSD Card Module Arduino - Random Nerd Tutorials

In-Depth Tutorial to Interface Micro SD Card Module with Arduino (lastminuteengineers.com)

What Time is It? (RTS DS3231)

As part of data logging and determining if the pump timer is correctly set, I need to know what time it is. To do this I will use a real-time clock module (RTC) with the DS3231 RTC chip and an AT24C32 EEPROM chip.

The RTC module communicates using the I2C interface for which the ESP32 has two build in IC2 controllers. The DS3231S RTC chip’s fixed I2C address is 0x68.

The test code can be found in the /WhatTimeIsIt folder and is pulled from the article In-Depth: Interface DS3231 Precision RTC Module with Arduino (lastminuteengineers.com). You will also need to load the uRTCLib by Naguissa library. The code can be used to set the chips time and then display the current time. Read the comments in the code for setting the current time.

Don’t Let Me Forget (AT24C32)

One of the nice things about the real-time clock module (RTC) is that in addition to the DS3231 RTC chip, it also has an AT24C32 EEPROM chip which can hold 32K of information even when the power is off. For this project I will use this memory to store settings like, during what time should the pump be on, the Wi-Fi SSID and password and user names, passwords and roles.

The test code can be found in the /DontLetMeForget folder and is also pulled from the article In-Depth: Interface DS3231 Precision RTC Module with Arduino (lastminuteengineers.com). You will also need to load the uEEPROMLib by Naguissa library. The code writes an integer, float, character, and string to the 24C32 EEPROM and then reads them back.

Show Me (OLED Display)

The last module to look at is an SSD1306 OLED 0.96 inch display with 128×64 pixels with an I2C interface. This module can be optional, but I added it because I’m not worked with one of these and I thought it would be useful to have some local visual feedback.

Just like the RTC and EEPROM the display connects through the I2C interface. One thing to note is that while the silk screen printing on the back of the module indicates an I2C address of 0x78, the correct address 0x3C.

The test code can be found in the /ShowMe folder and is pulled from the article In-Depth: Interface OLED Graphic Display Module with Arduino (lastminuteengineers.com). You will also need to load the Adafruit SSD1306, Adafruit GFX, and Adafruit Bus IO libraries. The code displays various different text messages on the display. You can find additional examples in the article ESP32 OLED Display with Arduino IDE - Random Nerd Tutorials.

More to Come…

If you can’t wait for the next article “v1.0 The Basics”, you can get a sneak peek at the GitHub repository.

Thus the birth of this project which will look at monitoring an existing mechanical system (AC sensing), logging usage data and alerting system failures all through a Wi-Fi Internet connection.

What is an Aerobic Septic System

An an Aerobic Septic System, also referred to as an Aerobic Wastewater Treatment System, is a small-scale sewage treatment system similar to a septic tank, but which uses an aerobic process (adding air or oxygen) for digestion rather than the anaerobic (without air) process used in septic systems.

Adding air promotes the growth of organisms that break down the solids, which are put through a clarifier and chlorinated for disinfection which produces a cleaner, more environmentally friendly discharge. This cleaner discharge eliminates the need for a drainage field (leach field) allowing the system to be installed on a small property where a standard septic tank system would not have been possible. Ref: Aerobic Septic Systems Explained - JT Septic Co - NE Oklahoma Septic Experts

Components

The first and obvious thing you are going to need is an ESP32 board. I chose to go with a development board as they are easy to prototype with, exposing all the pins along with a USB-UART bridge, reset- and boot-mode buttons, an LDO regulator and a micro-USB connector. I chose a ESP32-WROOM-32U. The U designates an external IPEX MHF4 antenna connector. While my aerobic septic control box (timers, alarms, etc.) is mounted on the house relatively close to the WiFi router many control boxes get mounted far from the house (100-200ft) at the aerobic tank. An external antenna in the design made more sense since I was still dealing with a a reasonable distance from the router and a concreate wall. Better safe, than sorry.

For an antenna I purchased a simple 8dBi antenna with a IPEX MHF4 to RP-SMA female pigtail cable that allowed for through hole mounting of the RP-SMA connector such that the antenna can be mounted outside a waterproof enclosure box. if you need more range I would suggest that you look at using a high gain Yagi directional antenna which can go up to 20dBi.

While building a PCB to mount the ESP32 and other components was appealing, this project is a prototype and many of the sensor components are not designed well to place on a PCB board. So, I decided it would be best to make all connections to the ESP32 through a breakout board. This will also make the design more modular should a component need to be replaced. Remember, I’m connecting sensors to AC circuits which are much more susceptible to power surges (i.e. lighting!).

Note that not all ESP32 development boards are equal when it comes to board width pin spacing. Some come in a 0.9” pin spacing, while others in a 1.0” pin spacing. The HiLetgo board I chose comes in a 1.0” pin spacing. Not realizing this and thinking that spacing was the same for all development boards I initially bought a breakout board with the smaller 0.9” pin spacing. The one suggested in following parts list will accept both the 0.9” and 1.0” pin spacing.

The basic function of this project is to measure if the power is on/off and if the pump is actually running. To determine if the power is on (i.e. the timer has switched on) I found some 220V optocoupler modules that can output a 3-5 volt TTL (digital) signal. Connected to any GPIO (general-purpose input/output) pin you get 0 for on and 1 for off.

While the power may be turned on, there is no guarantee that the pump is actually running since the pump circuit contains a low water level float switch designed to turn the pump off when the tank is empty. This means that I have to monitor the current flow in the circuit to determine if the pump is really running. I originally chose a cheaper current sensor that only measures 0-5 amps as I didn’t care about what the current was, I only needed to know current was being pulled. Turns out cheaper is not always better and after hours of trying to get a reading I decided to go with a current sensor that could read a full 20 amp range. The better sensor will also allow me to monitor the pump motor health by specifying an operational current range which could be used to indicate problems like a clogged intake or sprinkler outlet.

To log data I will need an SD card and real-time clock(RTC) to correctly timestamp the data. An Arduino compatible SD card reader with a SPI interface are extremity easy to find , as well as an I2C DS3231 real-time clock module. The real-time clock module suggested below also contains an AT24C32 EPROM memory chip which I will need to store things like the Wi-Fi SSID/password and administrator users/passwords. If you don’t buy a RTC with a memory chip you can add a separate memory module.

While it’s not necessary I thought it would be nice to add an OLED display. These are small, cheap and run on I2C. For this project the plan is to display things like the IP address (maybe even a QR code) and other useful status Information for when you open the enclosure to check/program.

To power the ESP32 and added modules I will need a USB wall charger and cable. The plan is to run power into the added enclosure using an extension cord with the female plug side in the box and male plug side cut off and spliced into the aerobic system power terminals. This way any standard USB wall charger can be plugged in to the female plug. The reason for the 6 foot cable is that it allows you to walk out to the septic system with your laptop and plug into ESP32 to diagnose and/or program as needed.

Parts List

Visual Studio Code

While the Arduino IDE 2 has come a long way from what is now referred to as the Arduino Legacy IDE or Arduino IDE 1, a lot of developers including myself prefer to use VS Code, short for Visual Studio Code. VS Code is an extremely versatile editor that through the implementation of extensions can work across multiple languages and platforms. To use VS code you will need to install the Arduino extension. You might also want to look at the PlatformIO IDE for VS Code extension.

More to Come…

Check out Part 2 - Beta Testing.

Typora is a new type of markdown editor. Where most all markdown editors, like VS Code have one window in which you type the plain text markdown and another window in which you can preview that text in a formatted view. Typora removes the preview window, mode switcher, syntax symbols of markdown source code, and all other unnecessary distractions. The result is a seamless experience in an editor that serves as both a reader and a writer. And yes, I’m using Typora to type this article.

If you do any kind of markdown editing, I strongly encourage you to download the editor for a 15-day trial and I guarantee you will be sold on paying $14.99. And did I mention, it runs on Mac, Linux and Windows.

Now when I say almost completed, I mean it’s only in pre-release because I have yet to complete the assembly documentation. I have been printing eggs for over 7 years and other than plans to write a simpler/user friendly version option for the non-hobbyist who has no desire to connect the SphereBot to a computer, the software is complete.

What’s an EggBot?

The EggBot is a compact, easy to use open-source art robot that can draw on spherical or egg-shaped objects. (ref: The Original EggBot - Simple, fun, & open source CNC art robot (egg-bot.com))

What is a SphereBot?

The SphereBot, an Eggbot clone. Since the conception of the EggBot hobbyist have been trying to build their own version of the EggBot. Just search Thingiverse for EggBot or SphereBot and you will find many projects.

What is version 2.0?

As for the version 2.0 designation, this code is a fork of the zaggo/SphereBot originally modified for the Adafruit Motor/Stepper/Servo Shield for Arduino v2 Kit in the jinschoi/SphereBot fork. This fork takes on a large number of changes including consistent naming conventions, an advanced G-Code parser, additional custom G-Code commands, a touchscreen control, SD card reader support and audio alerts.

So, please check out version 2.0 tgolla/SphereBot!

VS Code is a top shelf open-source editor that runs on Windows, Linux, and Mac with the IntelliSense you have come to love in Visual Studio, the ability to debug code and Git build built in.

Want more? Then just look at the at multitude of extensions in the VS Code Marketplace that make it a first-class editor. The following are just a few of the VS Code extensions you should check out.

• .NET Runtime Install Tool - Visual Studio Marketplace
• adr-tools - Visual Studio Marketplace
• Arduino - Visual Studio Marketplace
• Azure Account - Visual Studio Marketplace
• Azure AD B2C - Visual Studio Marketplace
• Azure Functions - Visual Studio Marketplace
• Azure Logic Apps (Consumption) - Visual Studio Marketplace
• Azure Repos - Visual Studio Marketplace
• Azure Resources - Visual Studio Marketplace
• Azure Terraform - Visual Studio Marketplace
• Better C++ Syntax - Visual Studio Marketplace
• C/C++ - Visual Studio Marketplace
• C/C++ Extension Pack - Visual Studio Marketplace
• C/C++ Themes - Visual Studio Marketplace
• C# - Visual Studio Marketplace
• CMake - Visual Studio Marketplace
• CMake Language Support - Visual Studio Marketplace
• CMake Tools - Visual Studio Marketplace
• Dev Containers - Visual Studio Marketplace
• Docker - Visual Studio Marketplace
• Doxygen Documentation Generator - Visual Studio Marketplace
• Durable Functions Monitor - Visual Studio Marketplace
• ESLint - Visual Studio Marketplace
• Git Graph - Visual Studio Marketplace
• Git History - Visual Studio Marketplace
• Git Tags - Visual Studio Marketplace
• GitHub Copilot - Visual Studio Marketplace
• GitHub Copilot Chat - Visual Studio Marketplace
• GitHub Pull Requests and Issues - Visual Studio Marketplace
• GitHub Repositories - Visual Studio Marketplace
• gitignore - Visual Studio Marketplace
• GitLens — Git supercharged - Visual Studio Marketplace
• Google Cloud Code - Visual Studio Marketplace
• HashiCorp Terraform - Visual Studio Marketplace
• HTML CSS Support - Visual Studio Marketplace
• ilspy-vscode - Visual Studio Marketplace
• isort - Visual Studio Marketplace
• Jekyll Run - Visual Studio Marketplace
• json - Visual Studio Marketplace
• Jupyter - Visual Studio Marketplace
• Jupyter Cell Tags - Visual Studio Marketplace
• Jupyter Keymap - Visual Studio Marketplace
• Jupyter Notebook Renderers - Visual Studio Marketplace
• Jupyter Slide Show - Visual Studio Marketplace
• Live Share - Visual Studio Marketplace
• Markdown All in One - Visual Studio Marketplace
• Markdown PDF - Visual Studio Marketplace
• Microsoft Edge Tools for VS Code - Visual Studio Marketplace
• nc-gcode - Visual Studio Marketplace
• PlatformIO IDE - Visual Studio Marketplace
• Postman - Visual Studio Marketplace
• PowerShell - Visual Studio Marketplace
• Prettier - Code formatter - Visual Studio Marketplace
• Print - Visual Studio Marketplace
• Pylance - Visual Studio Marketplace
• Python - Visual Studio Marketplace
• Remote - SSH - Visual Studio Marketplace
• Remote Explorer - Visual Studio Marketplace
• Remote Repositories - Visual Studio Marketplace
• Serial Monitor - Visual Studio Marketplace
• Thunder Client - Visual Studio Marketplace
• vscode-pets - Visual Studio Marketplace
• WSL - Visual Studio Marketplace
• XML Tools - Visual Studio Marketplace

Disclaimer: Remember VS Code is not an IDE like Visual Studio; it is an extremely powerful editor that at the minimum complements Visual Studio and with the multitude of extension comes very close to being a full fledged IDE. That said, check out PlatformIO IDE for VSCode which turns VS Code into a powerful embedded board development IDE for Arduino, ESP8266, ESP32, etc.

This example started with a search of the Internet, with the thought that surely someone else had thought of this exact thing. But to my dismay, the only thing I found that came remotely close to what I was looking for, I found in the GitHub repository Swashbuckle.AspNetCore.Filters published by Matt Frear. In the code repository is an AppendAuthorizeToSummaryOperationFilter which appends authorization information to the API summary. While this was close to what I was looking for I was concerned about space, some of the APIs I need to document can have 5-10 policies. I also wanted a bit more verbose description of the roles and/or policies required with the understanding that roles are ORed, while policies are ANDed. And I need it to handle the custom authentication attribute AuthorizeOnAnyOnePolicyAttribute. More about this attribute can be found in the GitHub repository TGolla.Swashbuckle.AspNetCore TGolla.AspNetCore.Mvc.Filters Readme.md file.

With an example to guide me I’ve put together the following operation filter which appends detailed information about authentication/authorization to the end of the API description. The description is the first thing you see following the summary and is populated with the text found inside the triple-slash comments <remarks></remarks> tags for the API call.

If you would like to see the filter in action check out the AppendAuthorizationToDescriptionExample website in the GitHub repository TGolla.Swashbuckle.AspNetCore or start using the filter in your project by installing the NuGet package TGolla.Swashbuckle.AspNetCore.

using Microsoft.AspNetCore.Authorization;
using Microsoft.OpenApi.Models;
using Swashbuckle.AspNetCore.SwaggerGen;
using TGolla.AspNetCore.Mvc.Filters;

namespace TGolla.Swashbuckle.AspNetCore.SwaggerGen
{
    /// <summary>
    /// Appends the API authentication/authorization information to the operation description.
    /// </summary>
    public class AppendAuthorizationToDescription : IOperationFilter
    {
        // Boolean used to determine if the AllowAnonymous description should be added.
        private bool excludeAllowAnonymousDescription = false;

        /// <summary>
        /// Initializes a new instance of the AppendAuthorizationToDescription class. 
        /// </summary>
        /// <param name="excludeAllowAnonymousDescription">Boolean used to determine if the AllowAnonymous description should be added.</param>
        public AppendAuthorizationToDescription(bool excludeAllowAnonymousDescription = false)
        {
            this.excludeAllowAnonymousDescription = excludeAllowAnonymousDescription;
        }

        /// <summary>
        /// Applys the appended API authentication/authorization information to the operation description.
        /// </summary>
        /// <param name="operation">An open API operation.</param>
        /// <param name="context">The operation filter context.</param>
        /// <remarks>Basic concepts pulled from https://github.com/mattfrear/Swashbuckle.AspNetCore.Filters.</remarks>
        public void Apply(OpenApiOperation operation, OperationFilterContext context)
        {
            if (context.GetControllerAndActionAttributes<AllowAnonymousAttribute>().Any())
            {
                if (!excludeAllowAnonymousDescription)
                    operation.Description += "\r\n\r\nAuthentication/authorization is not required.";

                return;
            }

            var authorizeAttributes = context.GetControllerAndActionAttributes<AuthorizeAttribute>();
            var authorizeOnAnyOnePolicyAttributes = context.GetControllerAndActionAttributes<AuthorizeOnAnyOnePolicyAttribute>();

            // Get list of authorize policies.
            List<string> authorizeAttributePolicies = authorizeAttributes.AuthorizeAttributePolicies();

            // Get a list of roles.
            List<string> authorizeAttributeRoles = authorizeAttributes.AuthorizeAttributeRoles();

            // Get list of authorize on any one policy policies. 
            List<string> authorizeOnAnyOnePolicyAttributePolicies = authorizeOnAnyOnePolicyAttributes.AuthorizeOnAnyOnePolicyAttributePolicies();

            if (authorizeAttributePolicies.Any())
                operation.Description += $"\r\n\r\nAuthorization requires {((authorizeAttributePolicies.Count > 1) ? "each of " : "")} the following {((authorizeAttributePolicies.Count > 1) ? "policies" : "policy")}: <b>{string.Join("</b>, <b>", authorizeAttributePolicies)}</b>";

            if (authorizeAttributeRoles.Any())
                operation.Description += $"\r\n\r\nAuthorization requires {((authorizeAttributeRoles.Count > 1) ? "any one of " : "")} the following {((authorizeAttributeRoles.Count > 1) ? "roles" : "role")}: <b>{string.Join("</b>, <b>", authorizeAttributeRoles)}</b>";

            if (authorizeOnAnyOnePolicyAttributePolicies.Any())
                operation.Description += $"\r\n\r\nAuthorization requires {((authorizeOnAnyOnePolicyAttributePolicies.Count > 1) ? "any one of " : "")} the following {((authorizeOnAnyOnePolicyAttributePolicies.Count > 1) ? "policies" : "policy")}: <b>{string.Join("</b>, <b>", authorizeOnAnyOnePolicyAttributePolicies)}</b>";

            if (authorizeAttributes.Any() && !authorizeAttributePolicies.Any() && !authorizeAttributeRoles.Any() && !authorizeOnAnyOnePolicyAttributePolicies.Any())
                operation.Description += "\r\n\r\nAuthentication, but no authorization is required.";
        }
    }
}

The code is relatively simple. The AppendAuthorizationToDescription operation filter is created using the IOperationFilter interface. The interface requires that you define the Apply() method. This method is handed an OpenApiOperation which gives you access to things like the API description and an OperationFilterContext which gives you access to the API attributes.

The first thing the Apply() method does is to look to see if the API method is decorated with an AllowAnonymousAttribute.

[AllowAnonymous]

This is done by calling the OperationFilterContext extension method GetControllerAndActionAttributes. This was pulled directly from the GitHub repository Swashbuckle.AspNetCore.Filters published by Matt Frear. The method takes an Attribute type and returns an IEnumerable list of any attributes of the type passed. If an AllowAnonymousAttribute is found and assuming the excludeAllowAnonymousDescription is false the message “Authentication/authorization is not required.” is appended to the operation description.

If an AllowAnonymousAttribute was not found the code continues on to again uses the GetControllerAndActionAttributes method, this time to collect a list of attributes type AuthorizeAttribute and a list of attributes type AuthorizeOnAnyOnePolicyAttribute. These lists are then queried using Linq to build string lists of policies, roles and authorize on any one policies with the extension methods AuthorizeAttributePolicies, AuthorizeAttributeRoles, and AuthorizeOnAnyOnePolicyAttributePolicies. The string lists are then used to generate a verbose message concerning the authorization required which is appended to the description and if there are no required policies, roles or authorize on any one policies the message “Authentication, but no authorization is required.” is returned.

To implement the filter requires that you call the OperationFilter<>() method with the AppendAuthorizationToDescription class inside AddSwaggerGen() in your progrms.cs file.

builder.Services.AddSwaggerGen(c =>
{
	...
    c.OperationFilter<AppendAuthorizationToDescription>();
	...
}

If you wish to exclude the AllowAnonymousAttribute message “Authentication/authorization is not required.” from an API description you will need to set the excludeAllowAnonymousDescription parameter argument to true.

c.OperationFilter<AppendAuthorizationToDescription>(true);

Also included in the TGolla.Swashbuckle.AspNetCore code repository is the AddSecurityRequirement operation filter which allows you to target operations that require a security schema.

builder.Services.AddSwaggerGen(c =>
{
	...
	c.AddSecurityDefinition("Bearer", new OpenApiSecurityScheme
    {
        Description = "JWT Authorization header using the Bearer scheme. Example: \"Authorization: Bearer {token}\"  A token can be acquired using any one of the /Tokens API calls.",
        Name = "Authorization",
        In = ParameterLocation.Header,
        Type = SecuritySchemeType.ApiKey,
        Scheme = "Bearer"
    });
   	...
}

When you define a security schema by invoking the AddSecurityDefinition method you also need to indicate which operations that scheme is applicable to. You can apply schemes globally (i.e. to ALL operations) through the AddSecurityRequirement method. This is what adds the Authorize button and unlock/lock icons to the end of each API summary.

builder.Services.AddSwaggerGen(c =>
{
	...
	c.AddSecurityRequirement(new OpenApiSecurityRequirement()
    {
        {
            new OpenApiSecurityScheme
            {
                Reference = new OpenApiReference
                {
                    Id = "Bearer",
                    Type = ReferenceType.SecurityScheme
                }
            },
            new List<string>()
        }
    });
    ...
}

Or you can be more specific by replacing the AddSecurityRequirement method with the AddSecurityRequirement operation filter provide in this example. The filter will only apply the security schema to API actions decorated with either the AuthorizeAttribute or AuthorizeOnAnyOnePolicyAttribute attributes. In this configuration it also makes sense to set the excludeAllowAnonymousDescription parameter argument to true.

builder.Services.AddSwaggerGen(c =>
{
	...
    c.OperationFilter<AppendAuthorizationToDescription>(true);
    ...
    c.OperationFilter<AddSecurityRequirement>(new OpenApiSecurityScheme
    {
        Reference = new OpenApiReference
        {
            Id = "Bearer",
            Type = ReferenceType.SecurityScheme
        }
    });
    ...
}

Addition information on adding security definitions and requirements can be found the Swashbuckle documentation domaindrivendev/Swashbuckle.AspNetCore: Swagger tools for documenting API’s built on ASP.NET Core (github.com).

Expresso is a full feature regular expression (RegEx) development tool that allows you to easily build and test your regular expressions. The tool allows you to create a project to build and test your specific regular expression which can be saved and modified in the future.

In design mode you can manually edit the regular expression; use an expression builder tool to create expression syntax by selecting criteria from a selection of tabs, checkboxes, and radio buttons; and provides an analyzer window that provide a treeview with an English translation of your regular expression.

In test mode you can provide a block of text on which you can execute and review the results of a match, partial match, exclude match, replace, validate and split. The tool also comes with a library of regular expression projects and will generate code in C#, Visual Basic, Managed C++ and C++/CLI.

Check it out at https://ultrapico.com/expresso.htm.

By default, when using Swashbuckle to generate a Swagger document, controllers are ordered alphabetically. There are however situations where alphabetical ordering is not best for your documentation. For example, consider an API project in which each controller represents a type of theater (Arena, Black Box, Proscenium, Thrust). Instead of sorting these alphabetically it makes more sense to order them as they are normally sorted when teaching theater (Proscenium, Thrust, Arena, Black Box).

In this article we will look at using a custom attribute to affect the order controllers are displayed in the Swagger documentation. Again, most of this code was adapted from Rob Janssen’s article Swashbuckle Custom Ordering of Controllers. A complete example can be found in the GitHub repository TGolla.Swashbuckle.AspNetCore and you can start using the SwaggerControllerOrder attribute in your project by installing the NuGet package TGolla.Swashbuckle.AspNetCore.

How It Works

When adding Swagger to your project using the Swashbuckle tools you have the ability to alter the sort order of the Controllers and even the individual APIs in the configuration information of the AddSwaggerGen method using OrderActionsBy. With OrderActionsBy you use a lambda expression to alter the sort key string used to order (group) API calls.

By default, the sort key is set to the first tag which by default, is set to the controller’s name, hence controllers are sorted alphabetically after which API calls are normally ordered by relative path. The following code represents the equivalent to the predefined default as defined using OrderActionsBy.

OrderActionsBy((apiDesc) => $"{apiDesc.ActionDescriptor.RouteValues["controller"]}");

In our example we are going to annotate each controller class with an order attribute, create a class that uses reflections to collect a list of all the controllers along with the order attribute value and expose a method in the class that returns the order and controller name in a sort key string. This method can then be used in the OrderActionsBy lambda expression.

OrderActionsBy((apiDesc) => $"{swaggerControllerOrder.SortKey(apiDesc.ActionDescriptor.RouteValues["controller"])}")

We will also look at going one step further by adding additional information like the HTTP method (GET, POST, PUT, DELETE,…) to affect the order of the individual API calls.

The Code

To begin we need to define an attribute class (SwaggerControllerOrderAttribute.cs).

namespace TGolla.Swashbuckle.AspNetCore.SwaggerGen
{
    /// <summary>
    /// Annotates a controller with a Swagger sorting order that is used when generating 
    /// the Swagger documentation to order the controllers in a specific desired order.
    /// </summary>
    /// <remarks>
    /// Ref: http://blog.robiii.nl/2018/08/swashbuckle-custom-ordering-of.html modified for AddSwaggerGen() extension OrderActionsBy().
    /// https://github.com/domaindrivendev/Swashbuckle.AspNetCore/blob/master/README.md#change-operation-sort-order-eg-for-ui-sorting
    /// </remarks>
    public class SwaggerControllerOrderAttribute : Attribute
    {
        /// <summary>
        /// Gets the sorting order of the controller.
        /// </summary>
        public uint Order { get; private set; }

        /// <summary>
        /// Initializes a new instance of the <see cref="SwaggerControllerOrderAttribute"/> class.
        /// </summary>
        /// <param name="order">Sets the sorting order of the controller.</param>
        public SwaggerControllerOrderAttribute(uint order)
        {
            Order = order;
        }
    }
}

This class allows us to annotate a controller with the attribute [SwaggerControllerOrder(x)] where x is an integer value indicating the order by ascending value. By default, a controller that is not annotated with the attribute is assigned the order value of 4294967295. Controllers with the same order assigned either by default or with the attribute are then sorted alphabetically, however this can be altered when setting the OrderActionBy method. In our example we will want to annotate each of our controllers as follows.

    [ApiController]
    [Route("theater/[controller]")]
    [SwaggerControllerOrder(1)]
    public class ProsceniumController : ControllerBase
    {
        // ...
    }

    [ApiController]
    [Route("theater/[controller]")]
    [SwaggerControllerOrder(2)]
    public class ThrustController: ControllerBase
    {
        // ...
    }

    [ApiController]
    [Route("theater/[controller]")]
    [SwaggerControllerOrder(3)]
    public class ArenaController: ControllerBase
    {
        // ...
    }

    [ApiController]
    [Route("theater/[controller]")]
    [SwaggerControllerOrder(4)]
    public class BlackBoxController: ControllerBase
    {
        // ...
    }

Next we need to create a class that will collect a list of controllers with the SwaggerControllerOrder attribute value.

using System.Reflection;

namespace TGolla.Swashbuckle.AspNetCore.SwaggerGen
{
    /// <summary>
    /// Class for determining controller sort keys based on the SwaggerControllerOrder attribute.
    /// </summary>
    /// <typeparam name="T">The type controllers should implement.  By default this would normally be ControllerBase or Controller
    /// unless you have derived your own specific api controller class.</typeparam>
    /// <remarks>
    /// Ref: http://blog.robiii.nl/2018/08/swashbuckle-custom-ordering-of.html modified for AddSwaggerGen() extension OrderActionsBy().
    /// https://github.com/domaindrivendev/Swashbuckle.AspNetCore/blob/master/README.md#change-operation-sort-order-eg-for-ui-sorting
    /// </remarks>
    public class SwaggerControllerOrder<T>
    {
        private readonly Dictionary<string, uint> orders;   // Our lookup table which contains controllername -> sortorder pairs.

        /// <summary>
        /// Initializes a new instance of the <see cref="SwaggerControllerOrder&lt;TargetException&gt;"/> class.
        /// </summary>
        /// <param name="assembly">The assembly to scan for for classes implementing <typeparamref name="T"/>.</param>
        public SwaggerControllerOrder(Assembly assembly)
            : this(GetFromAssembly<T>(assembly)) { }

        /// <summary>
        /// Initializes a new instance of the <see cref="SwaggerControllerOrder&lt;TargetException&gt;"/> class.
        /// </summary>
        /// <param name="controllers">
        /// The controllers to scan for a <see cref="SwaggerControllerOrderAttribute"/> to determine the sortorder.
        /// </param>
        public SwaggerControllerOrder(IEnumerable<Type> controllers)
        {
            // Initialize our dictionary; scan the given controllers for our custom attribute, read the Order property
            // from the attribute and store it as controllername -> sorderorder pair in the (case-insensitive) dicationary.
            orders = new Dictionary<string, uint>(
                controllers.Where(c => c.GetCustomAttributes<SwaggerControllerOrderAttribute>().Any())
                .Select(c => new { Name = ResolveControllerName(c.Name), c.GetCustomAttribute<SwaggerControllerOrderAttribute>().Order })
                .ToDictionary(v => v.Name, v => v.Order), StringComparer.OrdinalIgnoreCase);
        }

        /// <summary>
        /// Returns all <typeparamref name="TController"/>'s from the given assembly.
        /// </summary>
        /// <typeparam name="TController">The type classes must implement to be regarded a controller.</typeparam>
        /// <param name="assembly">The assembly to scan for given <typeparamref name="TController"/>s.</param>
        /// <returns>Returns all types implementing <typeparamref name="TController"/>.</returns>
        public static IEnumerable<Type> GetFromAssembly<TController>(Assembly assembly)
        {
            return assembly.GetTypes().Where(c => typeof(TController).IsAssignableFrom(c));
        }

        /// <summary>
        /// Determines the 'friendly' name of the controller by stripping the (by convention) "Controller" suffix
        /// from the name. If there's a built-in way to do this in .Net then I'd love to hear about it!
        /// </summary>
        /// <param name="name">The name of the controller.</param>
        /// <returns>The friendly name of the controller.</returns>
        private static string ResolveControllerName(string name)
        {
            const string suffix = "Controller"; // We want to strip "Controller" from "FooController"

            // Ensure name ends with suffix (case-insensitive)
            if (name.EndsWith(suffix, StringComparison.OrdinalIgnoreCase))
                // Return name with suffix stripped
                return name.Substring(0, name.Length - suffix.Length);
            // Suffix not found, return name as-is
            return name;
        }

        /// <summary>
        /// Returns the unsigned integer sort order value.  
        /// </summary>
        /// <param name="controller">The controller name.</param>
        /// <returns>The unsigned integer sort order value.</returns>
        private uint Order(string controller)
        {
            // Try to get the sort order value from our lookup; if none is found, assume uint.MaxValue.
            if (!orders.TryGetValue(controller, out uint order))
                order = uint.MaxValue;

            return order;
        }

        /// <summary>
        /// Returns an order key based on a the SwaggerControllerOrderAttribute for use with OrderActionsBy.
        /// </summary>
        /// <param name="controller">The controller name.</param>
        /// <returns>A zero padded 32-bit unsigned integer.</returns>
        public string OrderKey(string controller)
        {
            return Order(controller).ToString("D10");
        }

        /// <summary>
        /// Returns a sort key based on a the SwaggerControllerOrderAttribute for use with OrderActionsBy.
        /// </summary>
        /// <param name="controller">The controller name.</param>
        /// <returns>A zero padded 32-bit unsigned integer combined with the controller's name.</returns>
        public string SortKey(string controller)
        {
            return $"{OrderKey(controller)}_{controller}";
        }
    }
}

To use the class, we define an instance in the ConfigureServices method in the Program.cs file.

SwaggerControllerOrder<ControllerBase> swaggerControllerOrder = new SwaggerControllerOrder<ControllerBase>(Assembly.GetEntryAssembly());

When instantiated the SwaggerControllerOrder class searches the assembly for controllers of type ControllerBase and builds a dictionary list of controller names with their associated SwaggerControllerOrder attribute value. Those without the SwaggerControllerOrder attribute are not place in the list and will be assigned the default max value 4294967295.

With the class instantiated we can now use it when adding the Swagger generation service by adding an OrderActionsBy method call to the AddSwaggerGen configuration in the Program.cs file.

builder.Services.AddSwaggerGen(c =>
{
    c.OrderActionsBy((apiDesc) => $"{swaggerControllerOrder.SortKey(apiDesc.ActionDescriptor.RouteValues["controller"])}");
});

Taking It a Step Further

As mentioned earlier you can also use OrderActionsBy to affect the order of the individual API calls.

For example if you are adding group names (i.e. [ApiExplorerSettings(GroupName = "Hidden")]) to certain API calls to say hide those API calls from certain users you but don’t want then to be group (displayed) separately for your advanced users you can add the relative path to the sort key string.

c.OrderActionsBy((apiDesc) => $"{swaggerControllerOrder.SortKey(apiDesc.ActionDescriptor.RouteValues["controller"])}_{apiDesc.RelativePath}");

If you would like to see your API calls ordered alphabetically by HTTP method, you could use the following lambda expression.

c.OrderActionsBy((apiDesc) => $"{apiDesc.ActionDescriptor.RouteValues["controller"]}_{apiDesc.HttpMethod}");

If you would like to see API calls ordered by HTTP method, in a specific order you could add the following sort order array…

string[] methodsOrder = new string[5] { "get", "post", "put", "patch", "delete", "options", "trace" };

… with the following lambda expression.

c.OrderActionsBy(apiDesc => $"{apiDesc.ActionDescriptor.RouteValues["controller"]}_{Array.IndexOf(methodsOrder, apiDesc.HttpMethod.ToLower())}");

The possibilities are endless, and this example could even be taken one step further by adding an attribute to annotate IActionResult (API) calls.

Ref: https://github.com/domaindrivendev/Swashbuckle.AspNetCore#change-operation-sort-order-eg-for-ui-sorting

In the world of the Azure DevOps pipelines the easiest way to identify the code you deploy is by the pipeline build number.

What I’m going to talk about in the article is how you can output and then reference this build number with your deployed code.

The process requires that you first add a task to your pipeline YAML file that outputs the build number to a file in your artifacts which you can then access through your code. A complete example can be found on GitHub at https://github.com/tgolla/BuildNumbersInAzurePipelineArtifactsExample.

In your Azure pipeline YAML file (azure-pipelines.yml) you will need to add the following PowerShell task before the build task (DotNetCoreCLI@2... Command: ‘build’).

# Replace AzureBuildNumber.txt content with actual build number.
- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      New-Item $(Build.SourcesDirectory)/BuildNumbersInAzurePipelineArtifactsExample/AzureBuildNumber.txt -Force
      Set-Content $(Build.SourcesDirectory)/BuildNumbersInAzurePipelineArtifactsExample/AzureBuildNumber.txt '$(Build.BuildNumber)'

The task executes a PowerShell script that first creates (New Item) the AzureBuildNumber.txt file in the projects root folder and then adds (Set-Content) the build number. If the file already exists, the -Force parameter deletes the existing file and content before recreating the file.

In your code add the following method.

public static string AzureBuildNumber
{
    get
    {
        string azureBuildNumber = "Unknown";

        try
        {
            System.IO.StreamReader file = new System.IO.StreamReader("AzureBuildNumber.txt");

            azureBuildNumber = file.ReadLine() ?? "Unknown";
        }
        catch
        {
            // Do nothing.
        }

        return azureBuildNumber;
    }
}

The method will read the AzureBuildNumber.txt file from the projects deployed root folder and return the build number or ‘Unknown’ should the file be missing or empty. The complete example defines an AssemblyInfo class that also returns the assembly name and informational version.

Note: The assembly informational version is defined in the XML project file (BuildNumbersInAzurePipelineArtifactsExample.csproj) as part of the property group. By default, this is not automatically added to the project file and will need to be added manually or through the project’s properties tab under Package -> Package version: which must initially be changed to something other than 1.0.0.

<PropertyGroup>
  <TargetFramework>netcoreapp3.1</TargetFramework>
  <Version>1.0.1</Version>
</PropertyGroup>

You can find out more about the application at its GitHub repository (https://github.com/NuGetPackageExplorer/NuGetPackageExplorer). Scroll down to the README.md documentation for information on installing the application.