|
The Operations Center
The Mirago operations center is located at Global Switch in London's Docklands adjacent to the UK's main fibre backbone. Apart from excellent high speed connectivity into both the UK and mainland Europe, the center offers high levels of security and systems resilience. As Mirago grew, the move to this facility was a key milestone in our risk management strategy.
All servers in the data center are powered via comprehensive standby systems and are maintained in a fully regulated environment. The whole infrastructure is designed with resilience in mind and all units are monitored on a 24x7x365 basis.
The front end web server cluster which distributes the queries, operates Microsoft IIS .NET Servers. They communicate with the back end advertising and query servers via the Mirago transport layer to machines running Windows and Linux operating systems.
The Servers
The majority of the servers service the National web indexes. Most of these are housed in a 1U case and run Windows NT or Redhat Linux 9. One query server stores the index for up to 2 million documents. The larger National indexes will typically contain more than 100 million documents and are distributed on clusters of between 60 and one hundred machines.
That architecture gives Mirago ultimate scalability and fault tolerance. It means that one or more servers can be taken out of the cluster for maintenance without materially affecting the user experience. The transport layer, also allows system components to be located in different geographic locations.
When a query comes in from the Internet (and we often service well over one hundred per second) it is distributed by the webservers to the array of query servers. Each carry part of the National index and respond to the query separately. The query is also sent to the M3 Advertising servers which determine whether there are any Featured Sites which match that search. The responses are collated, the results are generated and the page is returned to the user. This entire process typically occurs in a few milliseconds.
The Robots
The 'robots' perform the role of retrieving and indexing documents which have changed since their last visit. The 'robots' are in fact a collection of machines which have access to terabytes of disk space to store the documents while the index is being built.
The indexing robots are the major users of bandwidth. They run on a continuous basis but are throttled in their usage of bandwidth by a software service called the Arbiter. This monitors the available bandwidth constantly and determines what percentage to use at specific times of the day.
| 
