Authentication improvements for testing your apps

We’ve just made it easier for developers to authenticate and test API calls with their own applications.

As the client owner, you can now authenticate with the password grant_type, allowing you to skip the authorization step of authenticating, and logging in with your WordPress.com username and password. You can also gain the global scope so that you no longer need to request authentication for each blog you wish to test your code with.

This is especially useful to contributors of the WordPress Android and iOS apps, which previously required special whitelisting on our part.

Here’s an example of how you can get started with using both these features:

Note that if you are using 2-step authentication (highly recommended) you will need to create an application password to be able to use the password grant_type.

$curl = curl_init( "https://public-api.wordpress.com/oauth2/token" );
curl_setopt( $curl, CURLOPT_POST, true );
curl_setopt( $curl, CURLOPT_POSTFIELDS, array(
    'client_id' => your_client_id,
    'client_secret' => your_client_secret_key,
    'grant_type' => 'password'
    'username' => your_wpcom_username,
    'password' => your_wpcom_password,
) );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1);
$auth = curl_exec( $curl );
$auth = json_decode($auth);
$access_key = $auth->access_token;

As noted above, these are only available to you as the owner of the application, and not to any other user. This is meant for testing purposes only.

You can review existing authentication methods here.

If you have any questions, please drop them in the comments or use our contact form to reach us.

A brand new Developer Site

As you may have noticed, we’ve just relaunched the WordPress.com Developer site (the very one you’re reading right now!) with a brand new look and feel!

We’ve rebranded the site to match the overall WordPress.com aesthetic as well as to align with the new user management and insights sections we launched just a few weeks ago.

E5A0rwxNNg-3000x3000

The goal of the redesign was not only to modernize the site but make it easier for you, our partners and third-party developers to find the information you are looking for. In addition, we’ve reviewed all of our existing documentation and past blog posts to make sure the information is accurate and relevant.

Over the next few months, you’ll see more updates to the site and more frequent blog posts from our team.

I’d personally like to thank the team that worked on the relaunch with me: Raanan, Kelly, Kat, Justin, and Stephane.

If you’d like to let us know what you think of the new site, report a bug, or have suggestions for future improvements, please comment below, tweet at us @AutomatticEng or contact us privately.

An efficient alternative to paging with SQL OFFSETs

Challenge

Running WordPress.com means having multimillion-record database tables. Tables which we often need to batch-query.

Provided we could hardly select (or update, etc) millions of records at once and expect speed, we commonly have to “page” our scripts to only handle a limited number of records at once, then move on to the next batch.

Classic, but inefficient, solution

The usual way of paging result sets in most SQL RDMS is to use the OFFSET option (or LIMIT [offset], [limit], which is the same).

SELECT * FROM my_table OFFSET 8000000 LIMIT 100;

But on a performance level, this means you’re asking your DB engine to figure out where to start from all on its own, every time. Which then means it must be aware of every record before the queried offset, because they could be different between queries (deletes, etc). So the higher your offset number, the longer the overall query will take.

Alternative solution

Instead, of keeping track of an offset in your query script, consider keeping track of the last record’s primary key in the previous result set instead. Say, its ID. At the next loop instance, query your table based on other records having a greater value for said ID.

SELECT * FROM my_table WHERE id > 7999999 LIMIT 100;

This will let you page in the same way, but your DB’s engine will know exactly where to start, based on an efficient indexed key, and won’t have to consider any of the records prior to your range. Which will all translate to speedy queries.

Here’s a real-life sample of how much difference this can make:

mysql> SELECT * FROM feeds LIMIT 8000000, 10;
  [...]
10 rows in set (12.80 sec)

mysql> SELECT * FROM feeds WHERE feed_id > 12958559 LIMIT 10;
  [...]
10 rows in set (0.01 sec)

I received the very same records back, but the first query took 12.80 seconds, while the alternative took 0.01 instead. :)

PHP/WordPress example

<?php
// Start with 0
$last_id = 0;

do {
    $blogs = $wpdb->get_results( $wpdb->prepare(
        'SELECT * FROM wp_blogs WHERE blog_id > %d LIMIT 100;',
        $last_id // Use the last ID to start after
    ) );

    foreach ( $blogs as $blog ) {
        // Do your thing!
        // ...
        // Record the last ID for the next loop
        $last_id = $blog->blog_id;
    }
// Do it until we have no more records
} while ( ! empty( $blogs ) );
?>

Like elasticsearch? We do too!

Elasticsearch tools

Elasticsearch, if you’re not familiar with it, is defined as a distributed restful search and analytics tool.

When it comes to implementing such an infrastructure, our developers not only face the challenges involved in indexing tens of millions of sites with grace and skill, they also write quite extensively about their related adventures, so others can benefit from their experiences.

You can find a plethora of posts on Greg Brown’s blog, under the appropriate tag. Subjects ranging from performance and scaling, all the way to “Elasticsearch, Open Source, and the Future“. And in true Automattician fashion, he isn’t even shy about recognizing his mistakes.

But Greg is not alone! Xiao Yu also recently wrote about the tools he uses, and a plugin he concocted for his own needs:

I’ve taken all that I wished I could do with both of those plugins and created a new Elasticsearch plugin that I call Whatson. This plugin utilizes the power of D3.js to visualize the nodes, indices, and shards within a cluster. It also allows the drilling down to segment data per index or shard. With the focus on visualizing large clusters and highlighting potential problems within. I hope this plugin helps others find and diagnose issues so give it a try.

How’s that for advanced? :)

Platform Updates: Batching Calls, Privacy Settings, and IDs

We’ve made a few more updates to our APIs recently that we wanted to share with you.

The biggest update is a new query parameter that’s now available on all endpoints. The new parameter allows you to batch certain calls together, so you only need to make one request to get related data instead of two or three.

Since we released our APIs, we’d always return a list of related endpoints in the “meta” response with a series of links:

"meta": {
        "links": {
            "self": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/21238",
            "help": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/21238\/help",
            "site": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907",
            "replies": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/21238\/replies\/",
            "likes": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/21238\/likes\/"
},

Now, by passing ?meta=site, you can automatically get the data from the above endpoints in the original response. Let’s take a look at an example.

Say you’re loading a specific post but you want to know the name and description of the site the post was on. You can do this by making a call to: https://public-api.wordpress.com/rest/v1/sites/en.blog.wordpress.com/posts/21238/?meta=site.

Which will give you a response like the following:

"meta": {
        "links": {
            "self": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/21238",
            "help": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/21238\/help",
            "site": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907",
            "replies": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/21238\/replies\/",
            "likes": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/21238\/likes\/"
        },
        "data": {
            "site": {
                "ID": 3584907,
                "name": "WordPress.com News",
                "description": "The latest news on WordPress.com and the WordPress community.",
                "URL": "http:\/\/en.blog.wordpress.com",
                "jetpack": false,
                "subscribers_count": 8396934,
                "meta": {
                    "links": {
                        "self": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907",
                        "help": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/help",
                        "posts": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/posts\/",
                        "comments": "https:\/\/public-api.wordpress.com\/rest\/v1\/sites\/3584907\/comments\/"
                    }
                },
                "is_private": false
            }
}

You can also pass multiple values in the meta query string. If you wanted the site endpoint and a list of likes for a post you can just pass "site,likes".

Two other updates we made are new responses:

  • We now include the value of privacy setting in the site information endpoint. A boolean value will be included as is_private.
  • We now include a global_ID response for all posts. This is a unique ID that you can use to identify posts if you are loading posts from multiple blogs in your application.

We hope you enjoy these updates. We’ll be making more improvements soon!

Originally posted on WordPress.com VIP:

One of the great things about developing for WordPress is the number of tools available for developers. WordPress core ships with a bunch of useful features (e.g. WP_DEBUG) with many more built by the community (like our own Rewrite Rules Inspector and VIP Scanner) that make development and debugging a breeze. The hardest part is getting your environment set up just right: knowing what constants to set, what plugins to install, and so on.

That’s why we built-in the Developer plugin. It’s your one-stop resource to optimally configure your development environment by making sure you have all the essential settings and plugins installed and available.

If you’re a WordPress developer, we highly recommend installing this plugin in your development environment. You can download the plugin from the WordPress.org Plugins Directory or directly from your WordPress Dashboard (Plugins > Add New).

Here’s a quick walk-through:

If you’d like…

View original 45 more words

Originally posted on Barry on WordPress:

Yesterday, Valentin Bartenev, a developer at Nginx, Inc., announced SPDY support for the Nginx web server. SPDY is a next-generation networking protocol developed by Google and focused on making the web faster. More information on SPDY can be found on Wikipedia.

At Automattic, we have used Nginx since 2008. Since then, it has made its way into almost every piece of our web infrastructure. We use it for load balancing, image serving (via MogileFS), serving static and dynamic web content, and caching. In fact, we have almost 1000 servers running Nginx today, serving over 100,000 requests per second.

I met Andrew and Igor at WordCamp San Fransicso in 2011.  For the next six months, we discussed the best way for Automattic and Nginx, Inc. to work together. In December 2011, we agreed that Automattic would sponsor the development and integration of SPDY into Nginx. The…

View original 250 more words