在Web页面里渲染Dot图形（Make Apache a GraphiViz Server）

在Web页面里渲染Dot图形（Make Apache a GraphiViz Server）

Gibbs Sampling of Latent Dirichlet Allocation Model in Go

As my first exercise on Google’s Go programming language, I wrote a Gibbs sampling training program of LDA using Go. This open source project is hosted on: http://code.google.com/p/lda-go, as a sister project of my previous LDA training and inference project in C++: http://code.google.com/p/ompi-lda/.

作为我学习Go语言的第一个练习，我用Go重写了LDA的Gibbs sampling训练程序：http://code.google.com/p/lda-go/。这个程序就作为我之前用C++写的并行LDA训练和推演程序（http://code.google.com/p/ompi-lda/）的姐妹项目吧。

Parallel LDA Gibbs Sampling Using OpenMP and MPI

I created an open source project, ompi-lda, on Google Code. This project is inspired by another LDA project which I initialized at Google and a recent parallel programming effort using OpenMP by xlvector.

I Moved My Blog to Wordpress

Since I found it is convenient to insert code and LaTeX math into my posts using Wordpress, I moved this blog to

http://cxwangyi.wordpress.com/

I may no longer update this blog.

Google Protocol Buffers 实用技术：解析.proto文件和任意数据文件

Google Protocol Buffers 是一种非常方便高效的数据编码方式（data serialization），几乎在Google的每个产品中都用到了。本文介绍 protocol buffers 的一种高级使用方法（在Google Protocol Buffer的主页上没有的）。

Protocol Buffers 通常的使用方式如下：我们的程序只依赖于有限的几个 protocol messages。我们把这几个 message 定义在一个或者多个 .proto 文件里，然后用编译器 protoc 把 .proto 文件翻译成 C++ 语言（.h和.cc文件）。这个翻译过程把每个 message 翻译成了一个 C++ class。这样我们的程序就可以使用这些 protocol classes 了。

但是还有一种不那么常见的使用方式：我们有一批数据文件（或者网络数据流），其中包含了一些 protocol messages 内容。我们也有定义这些 protocol messages 的 .proto 文件。我们希望解析数据文件中的内容，但是不能使用 protoc 编译器。

一个例子是 codex。codex是Google内最常用的一个工具程序。它可以解析任意文件中的 protocol message 内容，并且把这些内容打印成人能方便的阅读的格式。为了能正确解析和打印数据文件内容，codex 需要定义 protocol message 的 .proto 文件。

为了实现 codex 的功能，一种ad hoc的方法是：
1 把 codex 的基本功能（比如读取数据文件，打印文件内容等）实现在一个 .cc 文件里（比如叫做 codex-base.cc)
1 对给定的 .proto 文件，调用 protoc，得到对应的 .pb.h 和 .pb.cc 文件。
1 把 codex-base.cc 和 protoc 的结果一起编译，生成一个专门解析某一个 protocol message 内容的 codex 程序。
这个办法太ad hoc了。它为每个 .protoc 文件编写一个 codex 程序。

另一个办法是，如果我们有世界上所有的 .proto 文件，那么我们把它们都预先用 protoc 编译了，链接进 codex。显然这个搞法也是不现实的。

那么codex到底是怎么实现的呢？其实它利用了 protocol buffers 没有写入文档的一些 API。闲话少说，我们来看一段用这些“神秘的API“写的代码。这段代码用于解析任意给定的 .proto 文件：

#include <google/protobuf/descriptor.h>
#include <google/protobuf/dynamic_message.h>
#include <google/protobuf/io/zero_copy_stream_impl.h>
#include <google/protobuf/io/tokenizer.h>
#include <google/protobuf/compiler/parser.h>

//-----------------------------------------------------------------------------
// Parsing given .proto file for Descriptor of the given message (by
// name).  The returned message descriptor can be used with a
// DynamicMessageFactory in order to create prototype message and
// mutable messages.  For example:
/*
DynamicMessageFactory factory;
const Message* prototype_msg = factory.GetPrototype(message_descriptor);
const Message* mutable_msg = prototype_msg->New();
*/
//-----------------------------------------------------------------------------
void GetMessageTypeFromProtoFile(const string& proto_filename,
                              FileDescriptorProto* file_desc_proto) {
using namespace google::protobuf;
using namespace google::protobuf::io;
using namespace google::protobuf::compiler;

FILE* proto_file = fopen(proto_filename.c_str(), "r");
{
 if (proto_file == NULL) {
   LOG(FATAL) << "Cannot open .proto file: " << proto_filename;
 }

 FileInputStream proto_input_stream(fileno(proto_file));
 Tokenizer tokenizer(&proto_input_stream, NULL);
 Parser parser;
 if (!parser.Parse(&tokenizer, file_desc_proto)) {
   LOG(FATAL) << "Cannot parse .proto file:" << proto_filename;
 }
}
fclose(proto_file);

// Here we walk around a bug in protocol buffers that
  // |Parser::Parse| does not set name (.proto filename) in
  // file_desc_proto.
  if (!file_desc_proto->has_name()) {
 file_desc_proto->set_name(proto_filename);
}
}

这个函数的输入是一个 .proto 文件的文件名。输出是一个 FileDescriptorProto 对象。这个对象里存储着对 .proto 文件解析之后的结果。我们接下来用这些结果动态生成某个 protocol message 的 instance（或者用C++术语叫做object）。然后可以调用这个 instance 自己的 ParseFromArray/String 成员函数，来解析数据文件中的每一条记录的内容。请看如下代码：

//-----------------------------------------------------------------------------
// Print contents of a record file with following format:
//
//   { <int record_size> <KeyValuePair> }
//
// where KeyValuePair is a proto message defined in mpimr.proto, and
// consists of two string fields: key and value, where key will be
// printed as a text string, and value will be parsed into a proto
// message given as |message_descriptor|.
//-----------------------------------------------------------------------------
void PrintDataFile(const string& data_filename,
                const FileDescriptorProto& file_desc_proto,
                const string& message_name) {
const int kMaxRecieveBufferSize = 32 * 1024 * 1024;  // 32MB
  static char buffer[kMaxRecieveBufferSize];

ifstream input_stream(data_filename.c_str());
if (!input_stream.is_open()) {
 LOG(FATAL) << "Cannot open data file: " << data_filename;
}

google::protobuf::DescriptorPool pool;
const google::protobuf::FileDescriptor* file_desc =
 pool.BuildFile(file_desc_proto);
if (file_desc == NULL) {
 LOG(FATAL) << "Cannot get file descriptor from file descriptor"
            << file_desc_proto.DebugString();
}

const google::protobuf::Descriptor* message_desc =
 file_desc->FindMessageTypeByName(message_name);
if (message_desc == NULL) {
 LOG(FATAL) << "Cannot get message descriptor of message: " << message_name;
}

google::protobuf::DynamicMessageFactory factory;
const google::protobuf::Message* prototype_msg =
 factory.GetPrototype(message_desc);
if (prototype_msg == NULL) {
 LOG(FATAL) << "Cannot create prototype message from message descriptor";
}
google::protobuf::Message* mutable_msg = prototype_msg->New();
if (mutable_msg == NULL) {
 LOG(FATAL) << "Failed in prototype_msg->New(); to create mutable message";
}

uint32 proto_msg_size; // uint32 is the type used in reocrd files.
  for (;;) {
 input_stream.read((char*)&proto_msg_size, sizeof(proto_msg_size));

 if (proto_msg_size > kMaxRecieveBufferSize) {
   LOG(FATAL) << "Failed to read a proto message with size = "
              << proto_msg_size
              << ", which is larger than kMaxRecieveBufferSize ("
              << kMaxRecieveBufferSize << ")."
              << "You can modify kMaxRecieveBufferSize defined in "
              << __FILE__;
 }

 input_stream.read(buffer, proto_msg_size);
 if (!input_stream)
   break;

 if (!mutable_msg->ParseFromArray(buffer, proto_msg_size)) {
   LOG(FATAL) << "Failed to parse value in KeyValuePair:" << pair.value();
 }

 cout << mutable_msg->DebugString();
}

delete mutable_msg;
}

这个函数需要三个输入：1）数据文件的文件名，2）之前GetMessageTypeFromProtoFile函数返回的FileDescriptorProto对象，3）数据文件中每条记录的对应的protocol message 的名字（注意，一个 .proto 文件里可以定义多个 protocol messages，所以我们需要知道数据记录对应的具体是哪一个 message）。

以上代码中利用了 DescriptorPool 从 FileDescriptorProto 解析出 FileDescriptor（描述 .proto 文件中所有的 messages）。然后用 DynamicMessageFactory 从 FileDescriptor 里找到我们关注的那个 message 的 MessageDescriptor。接下来，我们利用 DynamicMessageFactory 根据 MessageDescriptor 得到一个 prototype message instance。注意，这个 instance 是不能往里面写内容的（immutable）。我们需要调用其 New 成员函数，来生成一个 mutable 的 instance。

有了一个对应数据记录的 message instance，接下来就好办了。我们读取数据文件中的每条记录。注意：此处我们假设数据文件中以此存放了一条记录的长度，然后是记录内容，接下来是第二条记录的长度和内容，以此类推。所以在上述函数中，我们循环的读取记录长度，然后解析记录内容。值得注意的是，解析内容利用的是 mutable message instance 的 ParseFromArrary 函数；它需要知道记录的长度。因此我们必须在数据文件中存储每条记录的长度。

接下来这段程序演示如何调用 GetMessageTypeFromProtoFile 和 PrintDataFile：

int main(int argc, char** argv) {
string proto_filename, message_name;
vector<string> data_filenames;
FileDescriptorProto file_desc_proto;

ParseCmdLine(argc, argv, &proto_filename, &message_name, &data_filenames);
GetMessageTypeFromProtoFile(proto_filename, &file_desc_proto);

for (int i = 0; i < data_filenames.size(); ++i) {
 PrintDataFile(data_filenames[i], file_desc_proto, message_name);
}

return 0;
}

Intel’s Single-chip Cluster Computer

I just read a post on Intel's single-chip cluster computer. This is really a unique thing: multiple cores in a chip, each core has its own private-and-fast memory: cache. Main memory becomes more like the NFS. Programmers write programs for this chip using MPI.

So, this chip is a cluster with super-fast network connection (64GB/s) and very limited memory (because they are implemented as cache).

How to Install Dropbox in Ubuntu Using SOCKS Proxy

As a Chinese, I live behind the G.F.W. and have to get access Internet through an SSH tunnel. However, the Dropbox installation procedure do not support either SOCKS5 or http proxy. (Yes, the document says it works with http_proxy environment variable, but seems it does not.) Thanks to two great tools: proxychains and tsocks, which can launch any application and handle their network communication request through a pre-configured proxy. With the help of these tools, I can install Dropbox on a brand new Ubuntu machine behind the G.F.W. Here follows the steps:

1. Install tsocks (or proxychains) under Ubuntu using synaptic. Thanks to Ubuntu, who made these tools standard packages.
2. Get a SSH tunnel (in any way you like). I paid for an account. So I can setup a proxy tunnel by the following command line on my computer:
   

   ssh -D 7070 my-username@my-service-provider-url

3. Configure your Web browser to use the tunnel. In Firefox, select to use SOCKS5 proxy: localhost:7070. This enables you access to Dropbox's homepage and download the Ubuntu package.
4. Install the package by clicking it in Nautilus. To check the installation, in a shell, type the command

   dropbox start -i

   If you can see some error messages complaining network access restriction, you made it.
5. Add the following lines to /etc/tsocks.conf:

   server = 127.0.0.1
   server_type = 5
   server_port = 7070

   If you are using proxychains, you need to modify /etc/proxychains.conf or make your own ~/.proxychains.conf.
6. This time, using tsocks to launch the Dropbox online install procedure:

   tsocks dropbox start -i

   Cross your fingers and wait for it to download and install Dropbox, until you see Dropbox icon appears on the top-right corner of your screen.
7. Right-click Dropbox icon, select "Preferences", and set SOCKS5 proxy like you did for Firefox. Hopefully, Dropbox starts to sync files you need now.

Good luck!

Building Distributed Programs Using GCC

In the case of distributed computing, a program is build (into a binary) and distributed on multiple computers for running. It is often that we do not want to install libraries depended by our program on every working computers. Instead, we can instruct GCC to link static libraries by setting the environment variable:

LDFLAGS=-static -static-libgcc

I have tried this method under Cygwin and Ubuntu Linux. However, if I do this under Darwin (Mac OS X 10.6 Snow Leopard), the linker complains that

ld: library not found for -lcrt0.o

In this Technical Q&A, Apple explains that they want to make Mac OS X upgrading easier, so they do not provide crt0.o to encourage dynamic linking.

Uint64 Constants

If we write uint64 a = 0xffffffffffffffff;, the compiler often complains:

error: integer constant is too large for ‘long’ type

What we need is to add the LLU suffix: uint64 a = 0xffffffffffffffffLLU;

Launch a Series of Hadoop Pipes Task

It is OK to call runTask multiple times from within main() in a Hadoop Pipes program to launch a series of MapReduce tasks. However, one thing to remember: MapReduce tasks must not have conflicts output directory, because Hadoop runtime does not start a MapReduce task whose output directory exists.

Incremental Parsing Using Boost Program Options Library

I have to say I fell in love with boost::program_options since the first time I use it in developing my own C++ MapReduce implementation. It can parse command line parameters as well as configuration files. This makes it convenient for programs which supports and expects a bunch of options, where a MapReduce program is a typical example.

A special use-case of a command line parser is that a function need to parse some options out from the command line parameters, and then the rest parameters are passed to another function, which parse other options. For example, the MapReduce runtime requires to get options like "num_map_workers", "num_reduce_workers", etc, and the rest of the program (user customized map and reduce functions) need to parse application-specific options like "topic_dirichlet_prior", "num_lda_topics", etc. boost::program_options supports such kind of multi-round parsing, where the key is boost::program_options::allow_unregistered(). Here attaches a sample program: (For more explanation on this program, please refer to the official document of boost::program_options.)

#include <iostream>
#include <string>
#include <vector>

#include <boost/program_options/option.hpp>
#include <boost/program_options/options_description.hpp>
#include <boost/program_options/variables_map.hpp>
#include <boost/program_options/parsers.hpp>

using namespace std;
namespace po = boost::program_options;

int g_num_map_workers;
int g_num_reduce_workers;

vector<string> foo(int argc, char** argv) {
 po::options_description desc("Supported options");
 desc.add_options()
   ("num_map_workers", po::value<int>(&g_num_map_workers), "# map workers")
   ("num_reduce_workers", po::value<int>(&g_num_reduce_workers), "# reduce workers")
   ;
 po::variables_map vm;
 po::parsed_options parsed =
   po::command_line_parser(argc, argv).options(desc).allow_unregistered().run();
 po::store(parsed, vm);
 po::notify(vm);

 cout << "The following options were parsed by foo:\n";
 if (vm.count("num_map_workers")) {
   cout << "num_map_workers = " << g_num_map_workers << "\n";
 }
 if (vm.count("num_reduce_workers")) {
   cout << "num_reduce_workers = " << g_num_reduce_workers << "\n";
 }

 return po::collect_unrecognized(parsed.options, po::include_positional);
}

void bar(vector<string>& rest_args) {
 po::options_description desc("Supported options");
 desc.add_options()
   ("apple", po::value<int>(), "# apples")
   ;
 po::variables_map vm;
 po::parsed_options parsed =
   po::command_line_parser(rest_args).options(desc).allow_unregistered().run();
 po::store(parsed, vm);
 po::notify(vm);

 cout << "The following options were parsed by bar:\n";
 if (vm.count("apple")) {
   cout << "apple = " << vm["apple"].as<int>() << "\n";
 }
}

int main(int argc, char** argv) {
 vector<string> rest_options = foo(argc, argv);

 cout << "The following cmd args cannot not be recognized by foo:\n";
 for (int i = 0; i < rest_options.size(); ++i) {
   cout << rest_options[i] << "\n";
 }

 bar(rest_options);
}

Finally I have to tell that early boost version (e.g., 1.33.1 packed in Cygwin) has bugs in program_options, which leads to core dump in case of unknown options. The solution to download and build your own boost libraries. I just built 1.43.0 on Cygwin on my Windows computer.

The Efficiency of AWK Associative Array

I did a little experiment comparing C++ STL map with AWK associative array in counting word frequency of large text files. The result is astonishing: AWK associative array is about 6 times faster than the C++ code!

At first, I put my eyes on the C++ code that splits a line into words. I tried STL istringstream, C strtok_r, and a string splitting trick that I learned in Google. However, these choices do not affect the efficiency saliently.

Then I realized the lesson I learned from parallel LDA (a machine learning method, which has been a key part of my research for over three years) --- map is about 10 times slower than map. I found that this issue has been thoroughly explained by Lev Kochubeevsky, principle engineer at Netscape, in 2003. Unfortunately, seems no improvement to STL string emerged since then.

On the other hand, I highly suspect that AWK, an interpreted language, implements a trie-based data structure for maps with string-keys.

SSHFS, Poor Guys' Network File-system

SSHFS is good: as long as one has SSH access, he can mount remote directories, even if he does not have administrative accesses required by traditional network filesystems like NFS or Samba.

Install and Configure MPICH2 on Ubuntu

The general procedure is described in
http://developer.amd.com/documentation/articles/pages/HPCHighPerformanceLinpack.aspx#four

I have encountered two cases where MPD on multiple nodes cannot communicate with each other:

1. The firewall prevents such communication, and
   
2. There are ambiguity in /etc/hosts.

For 1., we can disable iptables (the firewall commonly used in various Linux versions) and ufw (Ubuntu firewall) by the following commands:
sudo iptables -P INPUT ACCEPT
sudo iptables -P OUTPUT ACCEPT
sudo ufw disable

For 2., I just re-edit /etc/hosts to ensure all nodes are referred by their real IP addresses, instead of loop-back style addresses.

Hadoop Pipes Is Incompatible with Protocol Buffers

I just found another reason that I do not like Hadoop Pipes --- I cannot use a serialization of Google protocol buffer as map output key or value.

For those who are scratching your heads for weird bugs from your Hadoop Pipes programs using Google protocol buffers, please have a look at the following sample program:

#include <string>
#include <hadoop/Pipes.hh>
#include <hadoop/TemplateFactory.hh>
#include <hadoop/StringUtils.hh>

using namespace std;

class LearnMapOutputMapper: public HadoopPipes::Mapper {
public:
LearnMapOutputMapper(HadoopPipes::TaskContext& context){}
void map(HadoopPipes::MapContext& context) {
  context.emit("", "apple\norange\0banana\tpapaya");
}
};

class LearnMapOutputReducer: public HadoopPipes::Reducer {
public:
LearnMapOutputReducer(HadoopPipes::TaskContext& context){}
void reduce(HadoopPipes::ReduceContext& context) {
while (context.nextValue()) {
  string value = context.getInputValue(); // Copy content
      context.emit(context.getInputKey(), HadoopUtils::toString(value.size()));
}
}
};

int main(int argc, char *argv[]) {
return HadoopPipes::runTask(HadoopPipes::TemplateFactory<LearnMapOutputMapper,
                          LearnMapOutputReducer>());
}

The reducer outputs the size of the map output values, which contains special characters: new-line, null-term and tab. If Hadoop Pipes allows such special characters, then we should see reduce outputs 26, the length of string

"apple\norange\0banana\tpapaya".

However, unfortunately, we see 12 in the output, which is the length of string

"apple\norange"

This shows that map outputs in Hadoop Pipes cannot contain the null-term character, which, however, may appear in a serialization of protocol buffer, as explained in the protocol buffers encoding scheme at:

http://code.google.com/apis/protocolbuffers/docs/encoding.html

I hate Hadoop Pipes, a totally incomplete but released MapReduce API.

MPI-based MapReduce Implementation

离开Google之后就没有那么好用的MapReduce实现了。在拼命寻找替代品的时候，发现已经有人在用 MPI 实现 MapReduce。一个 open source 实现是：
http://www.sandia.gov/~sjplimp/mapreduce/doc/Manual.html

下载之后用了一下，发现对 MapReduce API 的封装很不到位。因此编程方法根本不是在Google 使用 MapReduce 时候的那一套。相比较而言，Hadoop Pipes 对 MapReduce API 的封装更便于使用。

这就牵扯到一个问题：MapReduce 到底强在哪儿？为什么用过 Google MapReduce 的人都会喜欢它？

MapReduce 的很多优势在论文里和论坛里都有人强调过了 —— 它可以处理海量的数据，可以支持 auto fault-recovery。但是我觉得最重要的一点是 MapReduce 的 API 很简单 —— 它容许程序员通过定义一个 map 函数和一个 reduce 函数就搞定一个并行程序。所有的 distributed IO、communications、task synchronization、load balancing、fault recovery都不用用户操心。

很多人在骂 Hadoop 慢。作为一个 Java implementation，Hadoop 确实是我见过的诸多 MapReduce 实现中最慢的（实际使用起来往往比 Last.fm 的 Bash MapReduce 还要慢），但是用Hadoop的人很多。难不成原因之一就是 API 好用？

我的感觉是：如果一个 MapReduce implementation 失去了 MapReduce 给程序员带来的便利，它的其他各种优势恐怕都要大打折扣了。(离开Google一个多月，我已经记不得我写过哪些不是用 MapReduce 的程序了。)

BTW：说到这里，顺便说一下， Sphere/Sector (http://sector.sourceforge.net/doc.html) 的 API 也不是 MapReduce API 。从 Sphere Sector 的 tutorial slides 里贴一下一个demo program：

Sector::init(); Sector::login(…)
SphereStream input;
SphereStream output;
SphereProcess myProc;
myProc.loadOperator(“func.so”);
myProc.run(input, output, func, 0);
myProc.read(result)
myProc.close();
Sector::logout(); Sector::close();

可以看到各种initialization、finalization、operator-loading 之类的操作都是需要用户来写的。其实把这些封装成 MapReduce API 并没有技术难度。而封装一下可以给用户省去很多麻烦。

Running Hadoop on Mac OS X (Single Node)

I installed Hadoop, built its C++ components, and built and ran Pipes programs on my iMac running Snow Leopard.

Installation and Configuration

Basically, I followed Michael G. Noll's guide, Running Hadoop On Ubuntu Linux (Single-Node Cluster), with two things different from the guide.

In Mac OS X, we need to choose to use Sun's JVM. This can be done using System Preference. Then In both .bash_profile and $HADOOP_HOME/conf/hadoop-env.sh, set the JAVA_HOME environment variable:
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home

I did not create special account for running Hadoop. (I should, for security reasons, but I am lazy and my iMac is only for personal development, but not real computing...) So, I need to chmod a+rwx /tmp/hadoop-yiwang, where yiwang is my account name, as well what ${user.name} refers to in core-site.xml.

After finishing installation and configuration, we should be able to start all Hadoop services, build and run Hadoop Java programs, and monitor their activities.

Building C++ Components

Because I do nothing about Java, I write Hadoop programs using Pipes. The following steps build Pipes C++ library in Mac OS X:

1. Install XCode and open a terminal window
2. cd $HADOOP_HOME/src/c++/utils
3. ./configure
4. make install
5. cd $HADOOP_HOME/src/c++/pipes
6. ./configure
7. make install

Note that you must build utils before pipes.

Build and Run Pipes Programs

The following command shows how to link to Pipes libraries:

g++ -o wordcount wordcount.cc \
-I${HADOOP_HOME}/src/c++/install/include \
-L${HADOOP_HOME}/src/c++/install/lib \
-lhadooputils -lhadooppipes -lpthread

To run the program, we need a configuration file, as shown by Apache Hadoop Wiki page.

Build libHDFS

There are some bugs in libHDFS of Apache Hadoop 0.20.2, but it is easy to fix them:

cd hadoop-0.20.2/src/c++/libhdfs
./configure
Remove #include "error.h" from hdfsJniHelper.c
Remove -Dsize_t=unsigned int from Makefile
make
cp hdfs.h ../install/include/hadoop
cp libhdfs.so ../install/lib

Since Mac OS X uses DYLD to mange shared libraries, you need to specify the directory holding libhdfs.so using environment variable DYLD_LIBRARY_PATH. (LD_LIBRARY_PATH does not work.):

export DYLD_LIBRARY_PATH=$HADOOP_HOME/src/c++/install/lib:$DYLD_LIBRARY_PATH

You might want to add above line into your shell configure file (e.g., ~/.bash_profile).

Get Through GFW on Mac OS X Using IPv6

In my previous post, I explained how to get through the GFW on Mac OS X using Tor. Unfortunately, it seems that Tor has been banned by GFW in recently months. However, some blog posts and mailing list claims that GFW has not been able to filter IPv6 packets. So I resorted to the IPv6 tunneling protocol, Teredo. A well known software implementation of Teredo on Linux and BSD is Miredo. Thanks to darco, who recently ported Miredo to Mac OS X, in particular, 10.4, 10.5 and 10.6 with 32-bit kernel. You can drop by darco's Miredo for Mac OS X page or just download the universal installer directly. After download, click to install, and IPv6 tunneling via IPv4 is setup on your Mac.

Before you can use IPv6 to get through the GFW, you need to know IPv6 addresses of the sites you want to visit. You must add these addresses into your /etc/hosts file, so the Web browser has no need to resolve the addresses via IPv4 (which is under monitoring by GFW). This Google Doc contains IPv6 addresses to most Google services (including Youtube).

Customizing Mac OS X 10.6 For a Linux User

I have been a Linux user for years, and changed to Mac OS X 10.6 Snow Leopard in recent months. Here follows things I've done for Snow Leopard to make it suit for my work habits.

• Emacs.
  I prefer Aquamacs version 20.1preview5 than the stable version 19.x when I wrote this post. Aquamacs has many useful Emacs plugins packed already, including AUCTeX for LaTeX editing.
  
• PdfTk.
  Under Unix, Emacs/AUCTeX invokes pdf2dsc (a component in the pdftk package) to do inline preview in PDFLaTeX mode. Under Mac OS X, thanks to Frédéric Wenzel, who created a DMG of PdfTk for us.
• LaTeX/PDF Preview.
  There is a free PDF viewer, Skim, under Mac OS X, which works like the ActiveDVI Viewer under Linux, but displays PDF files instead of DVI. Whenever you edit your LaTeX source and recompile, Skim will update automatically what it is displaying.
• Terminal.
  As many others, I use iTerm. To support bash shortcut keys like Alt-f/b/d, you need to customize iTerm as suggested by many Google search results. In particular, remember to select "High interception priority" when you do such customization for iTerm under Snow Leopard.

Chrome 的安全机制

今天看到多篇新闻报道：在 Pwn2Own 2010 黑客大赛上，针对各种浏览器的攻击中，只有 Google Chrome 屹立不倒。随便 Google 一下，会发现很多黑客把 Chrome 的安全性归结于 Chrome 的 sandbox（沙箱）机制。我因此好奇的看了看 Chromium（Google Chrome 的 open source project）的文档,

• http://www.chromium.org/developers/design-documents/sandbox
• http://dev.chromium.org/developers/design-documents/sandbox/Sandbox-FAQ

大概了解了一下 Chrome sandbox 的基本原理。

Chrome 会启动两类进程：target 和 broker：

1. Target 进程执行那些容易被黑客影响并做坏事的工作；主要包括（1）Javascript 程序的解释，和（2）HTML rendering。Target 进程是由 broker 进程创建的。创建之初，broker 就把 target 进程的各种访问权限都剥夺了。也就是说虽然 target 进程可以像所有用户态进程那样通过操作系统调用，请操作系统内核做事，但是操作系统内核会认为 target 进程没有权限，因而拒绝之。【注：在现代操作系统中，用户进程对任何系统资源的访问都得通过“请操作系统内核帮忙”来完成。】所以 target 实际上只能通过进程间调用，请 broker 进程来帮忙做事。
   
   
2. Broker 进程扮演着操作系统内核的角色 —— 因为 broker 进程执行的代码是浏览器的作者写的，并且不易被坏人注入坏代码，所以我们可以依赖它检查 target 进程请它做的事情是不是靠谱。如果不靠谱，则拒绝之。

简单的说，Chrome 的 sandbox 机制复制了操作系统的两层安全概念 —— 用户进程（target 进程）没实权，实权由操作系统内核（broker 进程）把持。实际上是装了第二把锁 —— 当用户和第三方软件对操作系统的错误配置导致操作系统安全机制失效的时候，第二把锁的作用就显示出来了。