No other species on Earth can play a symphony, form systems of government, and debate scientific issues with one another, quite like humans can. Social intelligence — including our capacities to understand each other’s minds and actions, and to collaborate and compete with each other — has played a key role in our progress as intelligent beings. Despite recent interests in building socially intelligent agents, there has been little work on developing benchmarks that systematically and rigorously evaluate the social intelligence of machine agents. In this talk, I will introduce two social AI challenges that we recently developed, each of which proposes a set of cognitively inspired tasks evaluating different aspects of machine social intelligence.