Thursday, March 28, 2024

Serverless GraphQL with Azure Functions, GraphQL for .NET, and Cosmos DB

 

Recently I've been tasked with building a PoC of Azure Functions based GraphQL service. I like tasks like this, especially if I can share my experience. I hope someone will benefit from this one.

Probably the most popular implementation of GraphQL for .NET is graphql-dotnet. It doesn't have a ready to use integration with Azure Functions, but it has a ready to use integration with ASP.NET Core. Taking into consideration that Azure Functions are built on top of ASP.NET Core and recently have been given support for dependency injection, that's the next best thing.

Building a GraphQL Azure Function

The core part of graphql-dotnet server implementation is available as a separated package: GraphQL.Server.Core. This is great. It gives us all the needed services without the strictly ASP.NET Core specific stuff (like middleware). This means that GraphQL set up for Azure Functions can be done in the same way as for ASP.NET Core (by registering dependency resolver, schema, services, and types).

[assembly: FunctionsStartup(typeof(Demo.Azure.Functions.GraphQL.Startup))]

namespace Demo.Azure.Functions.GraphQL
{
    public class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
            builder.Services.AddScoped<IDependencyResolver>(serviceProvider => new FuncDependencyResolver(serviceProvider.GetRequiredService));
            builder.Services.AddScoped<StarWarsSchema>();

            builder.Services.AddGraphQL(options =>
            {
                options.ExposeExceptions = true;
            })
            .AddGraphTypes(ServiceLifetime.Scoped);
        }
    }
}

This makes the IGraphQLExecuter service available to us. This is the heart of GraphQL.Server.Core. If we provide an operation name, query, and variables to the ExecuteAsync method of this service, it will take care of all the processing. So, we can inject that service and build our function based on it.

public class GraphQLFunction
{
    private readonly IGraphQLExecuter<StarWarsSchema> _graphQLExecuter;

    public GraphQLFunction(IGraphQLExecuter<StarWarsSchema> graphQLExecuter)
    {
        _graphQLExecuter = graphQLExecuter ?? throw new ArgumentNullException(nameof(graphQLExecuter));
    }

    [FunctionName("graphql")]
    public async Task<IActionResult> Run(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
        ILogger logger)
    {
        ...

        ExecutionResult executionResult = await _graphQLExecuter.ExecuteAsync(
            operationName,
            query,
            variables?.ToInputs(),
            null,
            req.HttpContext.RequestAborted
        );

        ...
    }
}

Now we are faced with two challenges. One is getting operation name, query, and variables out of request. The other is writing ExecutionResult to the response.

The challenging part of getting operation name, query, and variables out of request is that, depending on request method and content type, there are four different ways to do that. Putting all that code into the function would be unnecessary noise. This is why I've decided to extract that code into an extension method. This allows the function to be very clean.

public class GraphQLFunction
{
    ...

    [FunctionName("graphql")]
    public async Task<IActionResult> Run(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
        ILogger logger)
    {
        ExecutionResult executionResult = await _graphQLExecuter.ExecuteAsync(request);

        ...
    }
}

I'm not putting the code of the extension method here, as there is nothing really special about it. You can find it on GitHub.

When it comes to writing ExecutionResult to the response, graphql-dotnet gives us IDocumentWriter service. We could use this service in our function to write directly to the response and then return something weird like null or EmptyResult, but again this would be quite ugly. It's better to write dedicated ActionResult. There is no problem with accessing services from ActionResult, so entire logic can be nicely encapsulated.

internal class GraphQLExecutionResult : ActionResult
{
    private const string CONTENT_TYPE = "application/json";

    private readonly ExecutionResult _executionResult;

    public GraphQLExecutionResult(ExecutionResult executionResult)
    {
        _executionResult = executionResult ?? throw new ArgumentNullException(nameof(executionResult));
    }

    public override Task ExecuteResultAsync(ActionContext context)
    {
        if (context == null)
        {
            throw new ArgumentNullException(nameof(context));
        }

        IDocumentWriter documentWriter = context.HttpContext.RequestServices.GetRequiredService<IDocumentWriter>();

        HttpResponse response = context.HttpContext.Response;
        response.ContentType = CONTENT_TYPE;
        response.StatusCode = StatusCodes.Status200OK;

        return documentWriter.WriteAsync(response.Body, _executionResult);
    }
}

In result, the function is only a couple lines long (even with some basic errors logging).

public class GraphQLFunction
{
    ...

    [FunctionName("graphql")]
    public async Task<IActionResult> Run(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
        ILogger logger)
    {
        ExecutionResult executionResult = await _graphQLExecuter.ExecuteAsync(request);

        if (executionResult.Errors != null)
        {
            logger.LogError("GraphQL execution error(s): {Errors}", executionResult.Errors);
        }

        return new GraphQLExecutionResult(executionResult);
    }
}

Nice, let's see if it works. In my case, it didn't...

After preparing a request in Postman and sending it, as a response I received following error.

GraphQL.ExecutionError: Object reference not set to an instance of an object. ---> System.NullReferenceException: Object reference not set to an instance of an object.
    at GraphQL.DocumentExecuter.ExecuteAsync(ExecutionOptions options)

Well, this doesn't say much. I've spent some time going through DocumentExecuter code and concluded that the most likely cause of NullReferenceException is one if it's constructor dependencies being null. The DocumentExecuter has two constructors. One allows for providing dependencies and the other uses defaults. The AddGraphQL registers DocumentExecuter, but none of its dependencies. It would suggest that it expects that the parameterless constructor will be used, but it's not what is happening. As AddGraphQL uses TryAddSingleton to register DocumentExecuter, it should be enough to register it earlier and enforce the correct constructor.

[assembly: FunctionsStartup(typeof(Demo.Azure.Functions.GraphQL.Startup))]

namespace Demo.Azure.Functions.GraphQL
{
    public class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
            ...

            builder.Services.AddSingleton<IDocumentExecuter>(new DocumentExecuter());
            builder.Services.AddGraphQL(options =>
            {
                options.ExposeExceptions = true;
            })
            .AddGraphTypes(ServiceLifetime.Scoped);
        }
    }
}

Another attempt at sending a request and I saw the desired response. It works!

Adding Cosmos DB to the Mix

A database which goes well with Azure Functions is Azure Cosmos DB. It would be nice if we could use it in our GraphQL service. After all, it should only be a matter of registering DocumentClient as a service. It would also be nice if we could reuse the Azure Cosmos DB bindings configuration. I ended up with the below method.

public class Startup : FunctionsStartup
{
    public override void Configure(IFunctionsHostBuilder builder)
    {
        builder.Services.AddSingleton<IDocumentClient>(serviceProvider => {
            DbConnectionStringBuilder cosmosDBConnectionStringBuilder = new DbConnectionStringBuilder
            {
                ConnectionString = serviceProvider.GetRequiredService<IConfiguration>()["CosmosDBConnection"]
            };

            if (cosmosDBConnectionStringBuilder.TryGetValue("AccountKey", out object accountKey)
                && cosmosDBConnectionStringBuilder.TryGetValue("AccountEndpoint", out object accountEndpoint))
            {
                return new DocumentClient(new Uri(accountEndpoint.ToString()), accountKey.ToString());

            }

            return null;
        });

        ...
    }
}

It grabs the setting with connection string from IConfiguration, uses DbConnectionStringBuilder to get the needed attributes, and creates a new instance of DocumentClient. Now DocumentClient can be injected into query class and used to fetch the data.

internal class PlanetQuery: ObjectGraphType
{
    ...

    public PlanetQuery(IDocumentClient documentClient)
    {
        Field<ListGraphType<PlanetType>>(
            "planets",
            resolve: context => documentClient.CreateDocumentQuery<Planet>(_planetsCollectionUri, _feedOptions)
        );
    }
}

What Else?

As part of my PoC I've also experimented with a data loader, as it's crucial for GraphQL performance. I didn't play with more complex queries and mutations, but it doesn't seem like there should be any issue. You can find the complete demo here.

Saturday, March 2, 2024

Call Graph API from Azure Data Factory

Easy way to copy data from API to db using Azure data factory. In this exaple graphql end point used  to receive API data. 

Azure, Data Factory

Azure data factory is an ETL and orchestrator tool for building cloud-native data engineering pipelines. It has a lot of source connectors available and this list is growing rapidly. Microsoft has also enabled support for accessing Microsoft Graph API from Azure Data Factory. In this article, we will see how to call Graph API from Azure Data Factory using the managed identity.

What is Microsoft Graph API?

Microsoft Graph is a RESTful web API that enables you to access Microsoft Cloud service resources like Users, Groups, Mail, Calendars, Contacts, Files, etc. More information is available on the Microsoft docs.

Authentication

There are two ways to authenticate Graph API from Azure Data Factory.

Using Service principal

The service principle in simple terms is a service account. This enables programmatic access to resources in Azure. To create a service principal, we need to register an App in Azure Active Directory. This application has clientID, client Secret and app ID. These details can also be used to grant access to Microsoft Graph API.

Using Managed Identity

Managed identity is a service principal associated with resources in Azure. When we create a data factory in Azure, it automatically creates an app in Azure Active Directory. This means if we want to access Graph API using the data factory, we just need to grant Azure Data Factory app access to Graph API. This makes access management more secure and easy, isn’t it?

Let’s see it in action

This section describes how to call Graph API from Azure Data Factory using the managed identity. For demo purposes, we will get Azure active directory users’ data using Graph API and copy it to blob storage.

1. Providing Graph API access to Azure data factory.

a. Navigate to Azure active directory on Azure portal and search the data factory application (managed identity) under Enterprise applications.

b. Grant access to Data Factory app on Graph API as shown in the below image.

Note: Granting access to Graph API requires Azure active directory admin consent.

Azure data factory requires “user.read.all” permission to read users’ data from Graph API.

2. Setting up linked service for API calls.

a. Connect to the Azure portal and open the data factory. Create a new pipeline.

b. Create a new Linked service for REST API. Provide Authentication Type and AAD resource values as mentioned below.

Authentication Type : System Assigned Managed Identity

AAD resource : https://graph.microsoft.com/

3. Configure Copy Activity to get data from API and push it to blob Storage.

URL : https://graph.microsoft.com/v1.0/users

Pagination Rules : AbsolutionUrl = ['@odata.nextLink']

Execution of this ADF pipeline should generate a file in the target blob storage.

Pro tips:
1. Microsoft Graph API provides various API filtering parameters. More details can be found at this link.


Monday, February 19, 2024

How to use a GraphQL on Azure

📺 Watch the video : GraphQL on Azure (pens new window).

Basic template to start writing Graphql api using dotnet.

#Evolve your APIs with GraphQ

LGraphQL 

(opens new window)is a great way of creating and querying APIs. A GraphQL API is different from a REST API in that it allows the client application to query for certain fields of resources, like the name of a user, and only receive that data. The client controls what data it receives, not the server. Also, GraphQL provides an abstraction layer to the client, which means that clients don't need to query multiple URLs to access different data. GraphQL APIs get all the data a client needs in a single request when you use multiple GraphQL queries in one request.

GraphQL itself is not a framework or runtime that you can use. It is a specification of how to implement an API server-side and how to query it from a client application.

In this post, we'll create a GraphQL API in ASP.NET Core and run that on an Azure App Service Web App (opens new window).

#Prerequisites

If you want to follow along, you'll need the following:

#Run a GraphQL API in ASP.NET Core on Azure

Because GraphQL is a specification and not a framework, we can implement it by either creating our own, custom implementation, or by using an existing implementation. In this example, we'll use the existing implementation of the Hot Chocolate library (opens new window). This is a set of libraries that we can use as NuGet packages and support using GraphQL in ASP.NET Core. In fact, the sample application that we will use is the ASP.NET Core Star Wars example application that you can find here (opens new window). It is a simple GraphQL API that exposes information about Star Wars. Let's get started.

First, we will create the sample application on our local machine.

  1. Open a command prompt
  2. Create a new directory for the sample application and navigate to it, like this
mkdir starwars
cd starwars
1
2
  1. Create a new .NET Core application from the star wars example template with these commands:
dotnet new -i HotChocolate.Templates.StarWars
dotnet new starwars
1
2
  1. That's it! Open the folder of the sample application and open the solution file with Visual Studio.

(Star Wars example GraphQL application in Visual Studio)

The example application doesn't have any controllers and contains models and repositories and loads its data in-memory. We can explore what happens in the application by opening the Startup.cs file. In Startup.cs, we see that the file uses the HotChocolate libraries, which provide the magic ingredients to work with GraphQL.

using HotChocolate.AspNetCore;
1

And in the ConfigureServices method, we can see that the repositories are loaded and that we use an in-memory subscription provider, which loads the data for the GraphQL API in-memory. In a real-world scenario, you would use another data store that retrieves data from, let's say an Azure SQL Database (opens new window). Also, it uses the Hot Chocolate GraphQL ASP.NET Core middleware by using the AddGraphQL method and declaring a new GraphQL schema in it, using queries, mutations, subscriptions and types, like Human and Droid. This makes up the GraphQL API as it lays out which resources are available and which queries can access them. Drill down into any of these types to get more information.

        public void ConfigureServices(IServiceCollection services)
        {
            // Add the custom services like repositories etc ...
            services.AddSingleton<ICharacterRepository, CharacterRepository>();
            services.AddSingleton<IReviewRepository, ReviewRepository>();

            // Add in-memory event provider
            services.AddInMemorySubscriptionProvider();

            // Add GraphQL Services
            services.AddGraphQL(sp => SchemaBuilder.New()
                .AddServices(sp)
                .AddQueryType(d => d.Name("Query"))
                .AddMutationType(d => d.Name("Mutation"))
                .AddSubscriptionType(d => d.Name("Subscription"))
                .AddType<CharacterQueries>()
                .AddType<ReviewQueries>()
                .AddType<ReviewMutations>()
                .AddType<ReviewSubscriptions>()
                .AddType<Human>()
                .AddType<Droid>()
                .AddType<Starship>()
                .Create());
        }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

Finally, in the Configure method, the application sets up a route for the GraphQL endpoint and includes a playground, which we'll use later to test the API.

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app
                .UseRouting()
                .UseWebSockets()
                .UseGraphQL("/graphql")
                .UsePlayground("/graphql")
                .UseVoyager("/graphql");
        }
1
2
3
4
5
6
7
8
9
10
11
12
13
14

Let's see if we can get this application to run in Azure

  1. Right-click the project file in Visual Studio and select Publish
  2. In the wizard that opens, select Azure as the publish target and select Next

(Publish application window in Visual Studio)

  1. Next, select Azure App Service (Windows) and select Next
  2. Now, select "Create a new Azure App Service" as we are going to publish the app to a new App Service Web App
  3. In the Create new App Service window
    1. Leave the Name as it is
    2. Select a Resource group
    3. Create a new Hosting Plan
    4. Select Create. This can take a minute as it will create the App Service in Azure
  4. Once the App Service is created, select it and select Finish
  5. Now select Publish to publish the application to Azure. Once it is done, it will open the App Service URL in a browser

When the App Service URL is opened in a browser, it will show an error as there is nothing to display. Remember that we can use the GraphQL API with the route /graphql. So let's try that and let's go to the playground by adding /graphql/playground to the URL. This will open the playground, which is a visual GraphQL editor that you can use to test the API. In a real-world scenario, you would consume the API in an application by simply creating an HTTP request and firing it at the GraphQL endpoint.

(GraphQL playground running in Azure)

You can now test the API in the playground by creating queries. In the Docs tab on the right, you can find out which queries and resources are available for this API. Here is an example of a query that you can use:

query MyQuery{
    hero(episode:EMPIRE){
        name
    }
}
1
2
3
4
5

This query uses the hero query to retrieve character data for heroes that were featured in the EMPIRE episode of Star Wars. The fun thing of GraphQL is that you can define that you only want to retrieve certain fields, like the name of the character, and not the complete object.

#Conclusion

You can use implementations of the GraphQL (opens new window)specification to create APIs that allows clients to retrieve only the data that they need, without querying multiple URLs or dealing with API versions. And it is very easy to run a GraphQL API in Azure using an Azure App Service Web App (opens new window). Go and check it out!

How Netflix Scales its API with GraphQL Federation (Part 1)

  Netflix is known for its loosely coupled and highly scalable microservice architecture. Independent services allow for evolving at differe...