A Ruby Library for the Sistrix Web API

Ladies and Gentlemen, I present sistrix, a Ruby library for the Sistrix web API. The library supports all available API methods at the time of writing.

Get it as gem

gem install sistrix

or clone it from GitHub:

git clone http://github.com/elcamino/sistrix.git

How To Use It

require 'rubygems'
require 'sistrix'

Sistrix.config do |config|
   config.api_key = YOUR_API_KEY
end

seo_competitors = Sistrix.domain_competitors_seo(:domain => 'zeit.de', :num => 10)

credits = seo_competitors.credits
seo_competitors.competitors.each do |c|
  puts "domain: #{c.domain}"
  puts "match: #{c.match}"
end

visibility = Sistrix.domain_sichtbarkeitsindex(:domain => 'zeit.de')
visibility.sichtbarkeitsindex.each do |v|
  puts "domain: #{v.domain}"
  puts "date:   #{v.date}"
  puts "value:  #{v.value}"
end

Each API call is documented by the RSpec tests in the spec directory. Please refer to the API documentation at Sistrix web API for a list of available methods. An API method can be called by replacing the dot in its original name by an underscore (_):

domain.overview becomes Sistrix.domain_overview(:domain => ‘zeit.de’).

Posted in Uncategorized | Leave a comment

Block All IP Address Except For A Given Range To A Certain Port

You’ve got a server running to which you want to block access except for a given IP range. Here’s how to do this in Linux with iptables:

/sbin/iptables -X blockext

/sbin/iptables -N blockext
/sbin/iptables -A blockext -m iprange --src-range 202.45.4.0-202.45.4.255 -j ACCEPT
/sbin/iptables -A blockext -m iprange --src-range 202.45.4.0-202.45.4.255 -j ACCEPT
/sbin/iptables -A blockext -j DROP
/sbin/iptables -A INPUT -i eth0 -m tcp -p tcp --dport 80 -j blockext
/sbin/iptables -A INPUT -i eth0 -m tcp -p tcp --dport 443 -j blockext

These rules block all traffic to port 80 and 443 via eth0, in this case the external interface, but allows access via eth0 from 202.45.4.0-202.45.4.255.

Posted in Linux | Tagged , , | Leave a comment

Implement a Maintenance Page in Zend Framework

You need to switch your Zend Framework-based website into maintenance mode? Here’s how to do it. If the file ‘/tmp/maintenance.txt‘ exists on the webserver, a ZF project displays a maintenance page and sets the HTTP response code to “Service Temporarily Unavailable” (503).

First, create a controller plugin that stops further processing of the request if tmp/maintenance.txt exists and which calls the correct controller and action:

class VD_Controller_Plugin_Maintenance extends Zend_Controller_Plugin_Abstract
{
    protected $maintenance_file = '/tmp/maintenance.txt';

    public function preDispatch(Zend_Controller_Request_Abstract $request) {
        if (file_exists($this->maintenance_file)) {
            $request->setModuleName('default')
                    ->setControllerName('index')
                    ->setActionName('maintenance')
                    ->setDispatched(true);
        }
    }
}

In this case, the maintenance action resides in the IndexController, so you should add it:

class IndexController extends VD_Controller_Action
{
   // ...

    public function maintenanceAction()
    {
        $this->getResponse()->setHttpResponseCode(503);
    }
}

Search engines shouldn’t index the maintenance page so we set the HTTP response code to 503 and tell everybody that the service is temporarily unavailable.

I’ll leave writing the view in ‘views/scripts/index/maintenance.phtml‘ up to your imagination.

Finally, you need to enable the Maintenance plugin in your Bootstrap.php:

class VD_Application_Bootstrap extends Zend_Application_Bootstrap_Bootstrap
    public function _initPlugins()
    {
        $front = Zend_Controller_Front::getInstance();
        $front->registerPlugin(new MM_Controller_Plugin_Maintenance());
    }
}

The maintenance page functionality should work now. If you create ‘/tmp/maintenance.txt‘ the maintenance page will be displayed for every possible URL.

Posted in PHP | Tagged , , | Leave a comment

Send Emails in Ruby

If you ever need to send emails from ruby, use the mail gem. Run

gem install mail

to install the gem. Here’s how to send an UTF-8 encoded multipart mail via SMTP and TLS.

require 'mail'

m = Mail.new do
  delivery_method :smtp, {
    :address => 'smtp.mydomain.com',
    :domain => 'mydomain.com',
    :user_name => 'smtp_login',
    :password => 'smtp_password',
    :enable_starttls_auto => true,
    :authentication => :plain,
  }

  from    'me@mydomain.com'
  to      'you@yourdomain.com'
  subject 'Testing, 1, 2, 3!'

  text_part do
    content_type 'text/plain; charset=UTF-8'
    body 'Hi there!'
  end

  html_part do
    content_type 'text/html; charset=UTF-8'
    body '<html><body><h1>Hi there!</h1></body></html>'
  end
end

m.deliver
Posted in Ruby | Tagged , , | Leave a comment

Enable Caching In All Models

When your web project takes off and receives a lot of page impressions you’ll want to improve its performance. One way to achieve this is to reduce expensive SQL queries by caching the data your models retreive from the database.

Zend Framework offers a variety of ways to cache data…I’ll explain how you can add a cache to all of your models with a few lines of code.

Zend_Cache_Frontend_Class can be used to cache the results of static and instance methods of classes. We’ll add one such cache object to every model. The cache model could look like this. Please note that this class only caches instance methods.

class TVD_Cache_Model_Base {
    private $object;
    public $cache;
    private $class_methods = array();

    public function __construct($object) {
        $this->object = $object;

        foreach (get_class_methods($object) as $method) {
            $this->class_methods[$method] = true;
        }

        $frontend_name    = 'Class';
        $frontend_options = array(
          'lifetime' => 1800,
          'cached_entity' => $object,
          'cached_methods' => array('count', 'load'),
        );

        $backend_name = 'File';
        $backend_options = array(
            'cache_dir' => '/tmp/cache',
            'file_name_prefix' => get_class($this->object),
        );

        $this->cache = Zend_Cache::factory($frontend_name, $backend_name, $frontend_options, $backend_options);
    }

    /**
     * Sets the tags for a cached method
     *
     * @param array $tags Tags, with which the data will be cached to make
     *                    invalidating the data easier
     */
    public function setTagsArray($tags = array()) {
        $this->cache->setTagsArray(array_merge(array(get_class($this->object)), $tags));
    }

    public function __call($method, $args) {

        if($this->class_methods[$method]) {
            $data = $this->cache->__call($method, $args);

            return $data;
        }

        throw new TVD_Cache_Model_Base_Exception('Method ' . $method . ' does not exist in this class ' . get_class($this->object) . '.');
    }
}

class TVD_Cache_Model_Base_Exception extends Exception {}

Next, you create a class from which all your models inherit and which contains an instance of the cache class:

class TVD_Db_Table extends Zend_Db_Table
{
    protected $cache;

    public function cache() {
        if (! $this->cache) {
            $this->cache = new TVD_Cache_Model_Base($this);
        }

        return $this->cache;
    }

    public function setCacheTags($tags = array()) {
        $class_name = get_class($this);
        $tag_list = array($class_name);

        $this->cache->setTagsArray(array_merge($tag_list, $tags));
    }
}

You can now choose whether or not you want to use the cached method call. Suppose you’ve got a Comments object with a function getLatest() that fetches the latest comments and takes too much time:

class Comments extends TVD_Db_Table
{
   public function getLatest() {
      // expensive SQL calls that consume a lot of time
   }
}

You can now call the uncached and cached methods at your leisure:

$commentsClass = new Comments();

// no cache, fetches data from the database on every call
$data = $commentsClass->getLatest();

// cached, fetches data only once and caches it for how long you tell the cache to do so
$data = $commentsClass->cache()->getLatest();

This is only the start…setting the right tags and invalidating the cache is beyond the scope of this short article but you’ll have to figure that out yourself.

Posted in PHP | Tagged , , , | Leave a comment

Configure Zend Framework For More Than One Database

By default, a Zend Framework project has only one database configured, it can be configured to use an arbitrary number of databases, though. Use Zend_Application_Resource_Multidb to set up the database connections using a few lines of code.

The database connection is usually configured in app/config/application.ini and looks like this:

resources.db.adapter = "Pdo_Mysql"
resources.db.params.host = "localhost"
resources.db.params.username = "user1"
resources.db.params.password = "passwd1"
resources.db.params.dbname = "db1"
resources.db.params.charset = "utf8"

Replace this part with the following lines:

resources.multidb.db1.adapter = "Pdo_Mysql"
resources.multidb.db1.host = "localhost"
resources.multidb.db1.username = "user1"
resources.multidb.db1.password = "passwd1"
resources.multidb.db1.dbname = "db1"
resources.multidb.db1.charset = "utf8"
resources.multidb.db1.default = true

resources.multidb.db2.adapter = "Pdo_Mysql"
resources.multidb.db2.host = "localhost"
resources.multidb.db2.username = "user2"
resources.multidb.db2.password = "passwd2"
resources.multidb.db2.dbname = "db2"
resources.multidb.db2.charset = "utf8"

This instructs ZF to initialize both database connections and to make the first one (db1) the default connection for your models.

Next, add a new _init method to your Bootstrap.php file that writes the database connections to Zend_Session, from where you can access them anywhere in your code.

public function _initDatabases() {
    $resource = $this->getPluginResource('multidb');
    $resource->init();

    Zend_Registry::set('db1', $resource->getDb('db1'));
    Zend_Registry::set('db2', $resource->getDb('db2'));
}

Finally, you can now assign the db2 connection to models whose data resides there by creating a _setup method in each model:

protected function _setup() {
    $this->_db = Zend_Registry::get('db2');
    parent::_setup();
}
Posted in PHP | Tagged , , , | Leave a comment

OpenVPN Server On Two Ports

For some people, the default OpenVPN Port (1194) doesn’t work because they’re behind a firewall that blocks outgoing traffic on that port. In cases like this it might be unfeasible to change the port for all clients but OpenVPN can be run with the same keys on a different port that’s usually not blocked by firewalls (e.g. 80 or 443) on the same server.

First, create your server certificates as described on http://openvpn.net/index.php/open-source/documentation/howto.html, create a server configuration file, we’ll call it server.conf from now on, and install everything in the OpenVPN configuration directory (e.g. /etc/openvpn on Debian).

Create a client certificate and verify that the setup you just created works.

Once you’ve made sure that it works, copy /etc/openvpn/server.conf to /etc/openvpn/server_port_443.conf.

OpenVPN doesn’t allow two server configurations to reside in the same network. Suppose your default configuration lives in the 172.16.1.0/255.255.255.0 network and has the following configuration…

server 172.16.1.0 255.255.255.0

Change this line to

server 172.16.2.0 255.255.255.0

in /etc/openvpn/server_port_443.conf. It’s likely that port 443 is only available via tcp, make sure you change

proto udp

to

proto tcp

Restarting the OpenVPN server should start both configurations. If all connected clients should be able to connect to each other, a few more steps are required.

First, set up a route from each subnet to the other in each server configuration:

/etc/openvpn/server.conf needs the following lines to be added:

client-to-client
push "route 172.16.2.0 255.255.255.0 172.16.1.1"

and /etc/openvpn/server_port_443.conf respectively needs

client-to-client
push "route 172.16.1.0 255.255.255.0 172.16.2.1"

Make sure, the linux kernel forwards IP packets by adding

net.ipv4.ip_forward=1

to /etc/sysctl.config.

This is it, things should be working smoothly on Port 1194/UDP and 443/TCP now.

Posted in Linux | Tagged , | Leave a comment