Ok - So I've noticed twitter was not on my IM anymore. I regularly post via my gmail account on my rim. I didn't think much of this.
Marcel sent me an invite to plurk, and some posts on his lug-nuts list showed signs of twitter problems, so I decided to go over to the site to see what's what.
Seems they've been having issues for a while - I started off my journey to a "we're down" page. But! The blog is up, so I mossy on over there to see what's what.
I found the entry: It's Not Rocket Science, But It's Our Work - Here we have a link to the infamous status page. We also have some Q/A which yielded:
Q: Is it true that you only have a single master MySQL server running replication to two slaves, and the architecture doesn’t auto-switch to a hot backup when the master goes down? A: We currently use one database for writes with multiple slaves for read queries. As many know, replication of MySQL is no easy task, so we've brought in MySQL experts to help us with that immediately. We've also ordered new machines and failover infrastructure to handle emergencies.
Ok, Their having problems, according to the site. Maybe I can help them out, over to the jobs page and their looking for a Systems Eng:
Qualifications
- M.S. Computer Science or related field preferred
- Extensive experience building large-scale server applications
- Expert knowledge developing and debugging in Java and C/C++ on Unix
- Knowledge of python and ruby
- Experience with distributed systems, operating system internals, filesystems, compilers, threading models, and server architectures
- Disciplined approach to testing and quality assurance
- Great written communication and documentation abilities
The bolded jumps right out at me. Sheesh! No wonder their having problems. Maybe I should dig into their infrastructure a little more some day.
Quick scan via netcraft shows us their using Linux/Apache - good call. Still need more details.
wget can let me see the host responses with -S:
jason@homer:/# wget -S www.twitter.com --21:47:55-- http://www.twitter.com/ => `index.html' Resolving www.twitter.com... 128.121.146.100 Connecting to www.twitter.com|128.121.146.100|:80... connected. HTTP request sent, awaiting response... HTTP/1.1 301 Moved Permanently Date: Sat, 07 Jun 2008 01:48:20 GMT Server: Apache Location: http://twitter.com/ Cache-Control: max-age=300 Expires: Sat, 07 Jun 2008 01:53:20 GMT Vary: Accept-Encoding Content-Length: 227 Connection: close Content-Type: text/html; charset=iso-8859-1 Location: http://twitter.com/ [following] --21:47:56-- http://twitter.com/ => `index.html' Resolving twitter.com... 128.121.146.100 Connecting to twitter.com|128.121.146.100|:80... connected. HTTP request sent, awaiting response... HTTP/1.1 200 OK Date: Sat, 07 Jun 2008 01:48:22 GMT Server: hi Status: 200 OK P3P: CP="NOI DSP COR NID ADMa OPTa OUR NOR" X-Runtime: 0.01355 ETag: "0b7227441bc85f068424c5ba65b9eb54" Cache-Control: private, max-age=0, must-revalidate, max-age=300 Content-Type: text/html; charset=UTF-8 Content-Length: 7302 Via: 1.0 twitter-web044.twitter.com Expires: Sat, 07 Jun 2008 01:53:22 GMT Vary: Accept-Encoding Connection: close Length: 7,302 (7.1K) [text/html]
Going to www.twitter.com/ yeilded a 301 Moved to /index.html (good, did this right). But whats interesting is, they have clobbered apache (baddly) to report a different server type. The 301 shows "apache", but the following 200 for the index.html shows "hi".
Also, jugging from the "Via" line, we have a load ballancer before the web servers - I assumed as much, but lets dig [Pun Entended] into the dns.
;; ANSWER SECTION: www.twitter.com. 60 IN CNAME twitter.com. twitter.com. 60 IN A 128.121.146.100
Odd - only one IP address. Must have a VIP on front end routers. No big deal there.
Nothing else shows us about the backend that I can see - so far.
