La version française est disponible ici: Optimiser les performances et les fonctionnalités de son instance StatusNet
Previously about StatusNet
This is the second part of my StatusNet serie. Previous parts are available here:
- Part 1: Howto install your own StatusNet instance with PHP-FPM and Nginx
- Part 2: Optimize you own StatusNet instance for better performances and features
- Part 3: StatusNet plugin to improve frontend web performances
I'll use some configurations I already wrote about here:
- Nginx: automatic vhosts configuration with subdomains, SSL and authentication support - second version
- PHP-FPM, application server made by PHP
Anyway, you should be able to use all purely StatusNet related configuration, whatever you web server could be.
If you have installed your own StatusNet instance, you may have noticed that's… it's slow.
By default, all actions are processed in synchronous mode. As an exemple, subscriptions, publications, etc… are done within the same HTTP connection.
That means you may face some timeout mostly because of PHP max execution time value. The result is that your instance may not be fully consistent: half-staged subscriptions, publications not completely propagated, …
Hopefully, StatusNet allows you to fine-tune its configuration. Let's see how to do it.
Disable unused plugins
When installed, some plugins are enabled by default. Unfortunatly, this point is, as far as I've seen, not well documented. After some search, you find //identi.ca/main/version. Since you use the same software, you have the same URL on your instance.
All default-enabled plugins are not always usefull, at least from my point of vue. You can disable them in your config.php file, like this:
Depending on with plugin you consider, you will save some external requests (ex: GeoNames), some compute time (ex: RSSCloud) or web page load time (ex: Mapstraction).
Use asynchronous queue daemons
As I said in introduction, default StatusNet behaviour is to perform all tasks in a synchronous manner. Very simple, but not so optimized.
Let's say you have to publish a dent to all you followers spreaded accross identi.ca and many other standalone instances. That means your instance will connect to each other instances one time to deliver your dent. In a synchronous way, there are chances that the request ends with timeout without having dent delivered everywhere.
Of course, you still can increase PHP max execution time, but it's not really a solution since your web server will be more busy. The only suitable solution here is to have the job done as background job. That's exactly what queue are done for.
The drawback of this is you'll have to deal with a PHP daemon. you'll have to start the daemon at server boot eand be able to restart it in case of failure. That also means a shell access on the server which prevent you using StatusNet on most of the shared hosting platform.
Queue activation can be easily done:
MySQL will be used as default backend. For obvious security reasons, we'll use a specific non privileged user for running daemon. Here, statusnet user which is used for PHP-FPM StatusNet pool.
You can then start daemon. Since we have to switch user context, you need to start it as root:
Script will first check which daemon have to be started. Only one will be started by default: queuedaemon.php. You can also check it manually:
To stop daemons, you can use following command:
To get daemons started at server's boot, you can use /etc/rc.local:
Starting now, only subscriptions will be processed in a synchronous way. Dents propagation will be done asynchronously.
A benefic side effect of this is resilience, one more time. Should identi.ca's API become slow, your queue manager will try delivering dents again and again (default value is 10 tries).
Use PHP opcode caching extension
If you use PHP from some times, you know it for sure. In some cases, PHP will almost spend more time parsing and compiling scripts code than executing it.
I don't exactly know if it's the case here, but anyway I felt that adding APC support to PHP gives very interesting results. To enable APC extension:
According to my experience, default configuration for APC extension for PHP is enough for StatusNet. That is, one memory segment of 32MB.
Just beware of one thing: despite PHP-FPM and process using differents uid/gid, APC cache is shared between processes. Basically nothing special, but it's alway good to know that sort of thing.
Use data cache feature
StatusNet does support many caching systems: APC, Memcache, Memcached, Xcache, ... I'll only talk about the 2 I know better: APC & Memcache.
Basically, both APC and Memcache plugins intent to do the same thing: cache various informations, including configuration. Practically, the first which is enabled in configuration file wins.
You could want to use APC instead of Memcache to avoid TCP connections to memcache server. Yes, but... APC cache is not shared between PHP-FPM and PHP-CLI. That means that, if you use Queue daemons, you'll have 2 caches which won't synchronized.
Most visible effect: timeline won't get refreshed, even if new notices will be saved in database. Bad...
That's why I suggest to use Memcache plugin instead of APC. This way, both queue daemons and web interface will share the same cache.
You just have now to enable APC & Memcache support in StatusNet:
Of course, if not done, you must install memcache server:
Then you have to restart PHP so that changes can be applied:
Here you should see the difference… no ? OK so, let's add another cache layer. This is not a StatusNet wellknown plugin. There not so much informations about it internal behaviour but you can still have a look on source code. This plugin is called: InProcessCache
Just a important information though:
Note for later, always read comments… OK so, configuration becomes:
There are some other plugins related to cache operations. You can check them in /var/www/apps/statusnet/docroot/plugins directory.
Just beware to the fact that some of them may conflict as related PHP extensions (just like APC and XCache).
That said, a big thank to Karlesnine who helped me in understanding APC and Memcache StatusNet plugins behaviour.
Checkschema killed me
Checkschema is one of the most surprising default feature I've met in StatusNet. Its aim is to automatically check DB structure.
Some plugins might require specific MySQL tables and in a way, checkschema will really help you making sure, in background, that all tables are created. Really awesome when you discover StatusNet. But…
But by default, theses checks will take place… at each HTTP request !
Ouch, let's fix it quickly:
At the same time, I changed the way StatusNet connects to MySQL server. If you host all components ont he same server, you should always prefer sockets to TCP connections.
Of course, once checkschema has been disabled, you'll have to manually check DB structure and eventually make new tables created. For that, you should use the script checkschema.php as follow:
I still have no explanation about the default value accuracy for checkschema. Anyway, have it disabled was one on the most important performance increase I had with StatusNet, at least one of the most visible.
Tweaking user web interface
There are also some tips to optimize Web UI. To be honnest, you won't be really able to have full control on web page structure, there are no templates, but you still can separate domain name for static files.
As an example, in your profile page, most of your subscription can be displayed, which make each avatar downloaded one after the other. If you can not avoid huge number of downloads, you can spread them between differents domains:
Of course, you'll need a specific vhost definition for NGinx. Why a specific vhost ? Simply because I always disable access logs for statics resources, therefore limiting disk I/O from NGinx:
Finally, don't forget to separate static files from PHP scripts so that PHP script will never be stored in static root. If you forgot that, or if you prefer storing all files in the same directory, you may see your config.php leaked.
And, no it's not a bug. Simply, NGinx can not deal natively with PHP files, it will send them as plain text if you don't confiugre it another way.
Disable SMS support
StatusNet has been first designed to clone Twitter. Therefore, devs included SMs support. Since it's not very usefull for personnal instance, you can disable it:
Litlle tips I was very happy to discover: subscriptions are now much more easier because you won't need anymore to uncheck SMS, one subscription after the other.
How to find all these configuration options
As usual, all you need is code :)
More seriously, there are 2 files you need to know. First one initializes default config values which will be used if not declared anywhere:
The second one is a script which will give you all live config values:
This command is very usefull when hacking StatusNet, to understand what's going on. It's also usefull if you want to report an issue in StatusNet forums.
Here is the config.php file you should get if you applied all recipes I detailled here:
Here we're done with that part. Next time we'll see how to connect your StatusNet instance with Twitter and have all messges displayed in real-time.
Sources and references
- Getting started
- Official website
- Official website