APIs The Eloquent Way

Laravel solves almost all common use cases and provides best practices. Accessing data from your database? No problem, use eloquent! Validate user-data? Use FormRequests! Reading and writing data on AWS S3? Use flysystem!

Accessing third-party APIs? Use … wait a minute… Of course you can use Adam’s ZTTP Package or GuzzleHttp. But now you have to build all the logic for validation, transformation and so on yourself.

If only we could access APIs like we access the database. Isn’t there a better way? Let’s find out!

Motivation

Almost every project that we (as developers) work on nowadays has to integrate some third-party API. Whether you need a weather forecast for a specific date and location, the latest stock-market information, or INSERT_YOUR_USE_CASE_HERE, we have to interact with all kinds of APIs. (What was the last one you used?)

Even though APIs in general work in similar ways (see common standards), every one of them will have a unique set of endpoints and every endpoint will have a unique data structure.

Some APIs provide handy SDKs, often in the language of your choice. Amazon’s aws-php-sdk for example, offers high level access via an easy to use composer package. Unfortunately, not all APIs provide this luxury.

Think about the stock-market API from IEX. For quick access you can use ZTTP and simply request the list of available symbols in your controller.

1
2$symbols = \Zttp\Zttp::get("https://api.iextrading.com/1.0/ref-data/symbols")->json();
3

If you want to do a little bit more, you would have to create an abstraction layer that handles errors, validation and transformation. The further you dig in, the more logic and abstraction should be placed in this service layer. Maybe you want to enforce a data structure or further filter your data in a way that is not provided by the API. Collections would come in handy here, right? Then in your next project, you have to use a different API. Most of the API-request handling is the same. Copy and Paste?

As developers, we want a more abstract, reusable solution. Something that is more like the way we access our database. An Eloquent model provides a set of reusable methods and a place for your own entity related code (attributes, relationships etc.). It also provides an elegant way to build queries through method chaining.

1
2$user = User::where('email', $email)->first();
3

What if we could access an api like this?

1
2$symbol = IEX::Symbol()->where('symbol', $apiSymbol)->first();
3

That got us thinking. Hang on.

Solution

Our goal was to provide an interface similar to eloquent, so we built a reusable abstraction layer that implements an “API, Endpoint, Shape” pattern.

The JSONPlaceholder API for example has a base_url “https://jsonplaceholder.typicode.com” and provides a “/posts” endpoint that returns data (userId, id, title, body) in a certain shape.

• One API (JSONPlaceholder) is therefore an instance of ApiConsumer.
• One Resource (Posts) is an instance of Endpoint.
• The returned Post object is an instance of Shape.

Of course most APIs will have multiple Endpoints (and Shapes). The JSONPlaceholder API also provides a /comments Endpoint and a /users Endpoint, both with their own unique data-structure, or Shape. One ApiConsumer can have multiple Endpoints with Shapes.

The Endpoint offers the same method chaining capabilities as the Eloquent query builder and the Shape represents the data container of the Model.

Think about Endpoints as the place to tell your application how to query “all” users and how to “find” a specific user. Shapes represent the data-structure that is returned by an Endpoint.

To mimic eloquent even further, we want to return all data as a collection of their specific shape. That way a user can interact with that data, in the same native way he is used to with eloquent when accessing the database.

This is especially useful, when an API does not provide certain functionality, like filtering data for example. In laravel you can execute a “where”-clause through the query-builder and therefore on your database. Alternatively you can take advantage of the functionality that collections provide and execute a “filter”-method on a collection.

1
2$users = User::where('age', '>=', 21)->get();
3

‍‍‍

$users = User::all();
$users = $users->filter(function ($value, $key) {
    return $value->age >= 21
});

‍


In Laravel, eloquent models also provide extended functionality. Think about transformations with mutators. Functionality that is separated from the database itself.

1public function getFullNameAttribute()
2{
3    return "{$this->first_name} {$this->last_name}";
4}

The Shapes also offer a place for custom code. Think about a data-structure for users that have two fields “first_name” and “last_name”. Instead of placing that into your controller or frontend code, you can add a function that handles that for you.

In the same way, we can specify the data-structure that we expect in a shape and apply a check, to see if the data returned by the api fulfills those structural requirements. We can also normalize the structure by applying a transformation like changing a field “username” to “user_name”.

What if you want to get the author (user) for a specific post? The basic structure of the API is similar to how you would setup your database-tables. The posts have a field user_id, that references the id in users.

1
2$user = Post::first()->user;
3

We can mimic eloquents relations by utilizing custom implementations of hasOne() and hasMany() functions. That enables us to write clean queries, like this, for our JSONPlaceholder API:

1
2$user = JSONPlaceholder::Post()->first()->user();
3

Results

Ok... so what do we have so far?

• A Package black-bits/laravel-api-consumer
• Artisan commands that generate the stubs for ApiConsumers, Endpoints and Shapes
• Model resources for an api in Endpoints and define their structure in Shapes
• A way to setup relations between resources (Endpoints)
• Access to an API, the same way you access your database with eloquent
• All collection methods chainable on Endpoints
• Our own custom collection methods to use on Endpoints
• Everything as a collection (Read HERE why collections are awesome)
  
  
• BONUS - Caching out of the box

Disclaimer

Even though we are pretty proud about what we accomplished over the last weekend, this package is not ready for production yet. Feel free to give it a try though.

Let's checkout the showcase, shall we?

Show Case

Are you still with us? Ok good. It's more fun from here on. We built a show case website to play around with the package. You can download the show case project here, and the package is on packagist.org here, if you want to try it out yourself. (Design was obviously not our focus - Adam and Steve have awesome tutorials on Design)

• We have implemented two simple APIs (IEX Trading API and JSONPlaceholder)
• The IEX Trading API does not offer full REST capabilities. We make use of our CollectionCallbacks to filter/search Companies by Smybol or Name
• JSON Placeholder API offers a straight forward REST API that returns demo data. We have implemented a where Method in our Endpoint
• All the API related code is located in the app/ApiConsumers directory
• The SymbolEndpoint has caching enabled (10 min)

ApiConsumers Setup

So what do i need to do to access the IEX API? Easy... just require our package and publish the configuration.

1composer require black-bits/laravel-api-consumer
2
3php artisan vendor:publish --provider="BlackBits\ApiConsumer\ApiConsumerServiceProvider"

Now, lets create our first ApiConsumer..

1
2php artisan make:api-consumer IEX
3

... and our first Endpoint (the Shape will get created automatically)

1
2php artisan make:api-consumer-endpoint Symbol -c IEX
3

Add the API Base URL to your configuration (/config/api-consumers.php)

1return [
2    'IEX' => [
3        'apiBasePath' => 'https://api.iextrading.com/1.0',
4    ],
5];

Your previously created ApiConsumer Class should look like this (/app/ApiConsumers/IEX/IEX.php)

1namespace App\ApiConsumers\IEX;
2
3use BlackBits\ApiConsumer\ApiConsumer;
4
5/**
6 * Class IEX
7 * @package  App\ApiConsumers\IEX
8 */
9class IEX extends ApiConsumer
10{
11    /**
12     * @return  \Illuminate\Config\Repository|mixed
13     */
14    protected function getEndpoint()
15    {
16        return config('api-consumers.IEX.apiBasePath');
17    }
18}

‍


Almost done ... let's define the resources in the previously created Symbol Endpoint (/app/ApiConsumers/IEX/Endpoints/SymbolEndpoint.php) $path specifies the url part on how to access the resource. With $shouldCache and $cacheDurationInMinutes you can define if API calls should be cached and for how long. The function all() defines how you expect to access all objects for that Endpoint.

1namespace App\ApiConsumers\IEX\Endpoints;
2
3use BlackBits\ApiConsumer\Support\Endpoint;
4
5/**
6 * Class SymbolEndpoint
7 * @package  App\ApiConsumers\IEX\Endpoints
8 */
9class SymbolEndpoint extends Endpoint
10{
11    /**
12     * @var  string
13     */
14    protected $path = '/ref-data/symbols';
15
16    /**
17     * @var  bool
18     */
19    protected $shouldCache = true;
20
21    /**
22     * @var  int
23     */
24    protected $cacheDurationInMinutes = 10;
25
26    /**
27     * @return  \Illuminate\Support\Collection|\Tightenco\Collect\Support\Collection
28     * @throws  \Exception
29     */
30    public function all()
31    {
32        return $this->get();
33    }
34}

‍


Technically we're good to go, but let's check the Shape (/app/ApiConsumers/IEX/Shapes/SymbolShape.php). Everything in here is optionally and the automatically created one is all you need. You can however set $return_shape_data_only if you only want to work with specific fields. You can also set $require_shape_structure to add a check if the structure of the api result is what you would expect. $transformations let's you normalize fields (iexId => iex_id) and function company() defines a relation.

1namespace App\ApiConsumers\IEX\Shapes;
2
3use App\ApiConsumers\IEX\Endpoints\CompanyEndpoint;
4use BlackBits\ApiConsumer\Support\BaseShape;
5
6/**
7 * Class SymbolShape
8 * @package  App\ApiConsumers\IEX\Shapes
9 */
10class SymbolShape extends BaseShape
11{
12    /**
13     * @var  bool
14     */
15    protected $return_shape_data_only = true;
16
17    /**
18     * @var  bool
19     */
20    protected $require_shape_structure = true;
21
22    /**
23     * @var  array
24     */
25    protected $transformations = [
26        'symbol'    => 'symbol',
27        'name'      => 'name',
28        'date'      => 'date',
29        'isEnabled' => 'is_enabled',
30        'type'      => 'type',
31        'iexId'     => 'iex_id'
32    ];
33
34    /**
35     * @var  array
36     */
37    protected $fields = [
38        'symbol',
39        'name',
40        'date',
41        'is_enabled',
42        'type',
43        'iex_id'
44    ];
45
46    /**
47     * @return  mixed
48     */
49    public function company()
50    {
51        return $this->hasOne(CompanyEndpoint::class, 'symbol');
52    }
53}

‍


Usage

From here on you can query all Symbols from the IEX API like this:

1
2$symbols = IEX::Symbol()->all();
3

Let's say you want to get the related company information:

1$symbols = IEX::Symbol()->all();
2$company = $symbols->first()->company();
3
4// or
5$company = IEX::Symbol()->first()->company();

‍


Ok, great ... remember that screenshot from the beginning? What code was necessary in the controller to make it work? Here it is:

1/**
2 * @return  \Illuminate\Http\Response
3 */
4public function list(Request $request)
5{
6    $symbols = IEX::Symbol();
7
8    if (!empty($request->get('symbol'))) {
9        $symbol = $request->get('symbol');
10        $symbols->filter(function ($value, $key) use ($symbol) {
11            return str_contains($value->symbol, $symbol);
12        });
13    }
14
15    if (!empty($request->get('name'))) {
16        $symbols->filterName($request->get('name'));
17    }
18
19    $data = [
20        'symbols' => $symbols->take(10)->get()
21    ];
22
23    debug($data);
24    return response()->view('symbols.list', $data);
25}

‍


Let's check out another example with the JSONPlaceholder API. Here we want to get a specific Post with all related Comments.

And the only code necessary in our controller is this. The ApiConsumer-, Endpoint- and Shape-Baseclasses take care of the rest. Caching, relations, transformations, validating and enforcing responses and so on.

1/**
2 * @return  \Illuminate\Http\Response
3 */
4public function listComments($post)
5{
6    $post = JSONPlaceholder::Post()->where('id', $post)->first();
7
8    $data = [
9        'comments' => $post->comments(),
10        'post'     => $post
11    ];
12
13    debug($data);
14    return response()->view('json-placeholder.comments-list', $data);
15}

‍


Feel free to checkout the code in the repository in detail and let us know what you think?

Package Deep Dive

Interested in a technical deep-dive on what is going on under the hood? Drop us a note at hello@blackbits.io.

Next Steps

There is still a lot of work. Right now, there is no support for authentication. That is probably the next thing on our agenda. Afterwards we will implement other http methods (post / put /…) and improve the relation functionality. And of course ... testing.

Long term it could be possible to implement an auto-generation for standardized endpoints (JSON API / Swagger).

Conclusion

If you made it all the way down here, thanks you for your attention!

We certainly had a lot of fun going through ideas at the drawing board and exploring different implementations over the course of the weekend.

We will definitely pursue this further, as we believe that a standardized way to access APIs is still missing from the Laravel ecosystem.

We would love to get your input

• What do you think about the code division into the Consumer -> Endpoint -> Shape structure?
• Any important features we are missing?
• What kind of APIs are you using the most?
• What kind of APIs give you the most trouble implementing?