more on AI and (contemporary) cultural heritage: poeme electronique, RAI, and robocup

In the comment on the previous post on AI & cultural heritage, my brother wondered if something concrete would come from using the variety of (AI) techniques. Apart from the ones mentioned in that post that are being implemented in Italy and Haifa, there’s also one – working! – closer to home: the Poeme Electronique (virtual reality), and others, such as TV genre classification (neural networks) and Robocup (game theory).

Following up on the previously mentioned VR & cultural heritage, a VR version to regain lost art, and a documentary about the making-of, is available online for free, but you need a lot of bandwidth. It has been developed in the Virtual Poeme Electronique project led by Vincenzo Lombardo. The Electronic Poem was developed for & by Philips Eindhoven for the 1958 international exposition in Brussels to demonstrate its technology for society (as opposed to use technology for war) and thereby the first multimedia project of the electronic era. Unfortunately, it was broken down 2 months after the exhibition hosted 2 million visitors, and lost – other than various bits and pieces to build it, such as the sketches of Le Corbusier, music scores by Edgar Varese, the hyperbolic paraboloids drawings by Xenakis (the plan resembles a stomach, as metaphor that expo visitors would be ‘digested’ by the multimedia presentation in the building), and an old Philips video; see the materials section. The VR reconstruction is interesting from, a.o, a technological viewpoint: “the unity of this installation, that conveys images and sound paths of a great complexity in a common digital space, requires challenging solutions in the integration of the various media (design of the digital space, display of the visual show, organization of the sound paths) and in the interaction between the spectator and the installation” (copied from the project website); see also a summary on the fascinating cultural historical & artistic aspects. In the upcoming years, the expo will be rebuilt by Philips at the site of the former headquarters in Eindhoven.

Of course one could argue if TV is, or approximates, contemporary art. Instead of one’s subjective view, one could investigate this in a more structured fashion by analyzing the archives of the various types of programmes. However, e.g. RAI (the Italian public radio and TV organization) already has some 560 000 hours TV archive and 700 000 hours of radio archive. One step in the structuring of the archive to query and analyse it better is TV genre classification. [1] used a feed-forward neural network after the feature extraction process. Features of the video files are, a.o., luminance colour histograms, texture signature, average speech rate, and displaced frame difference. It works for the basics categories, such as news and talk shows, but I wonder about more recent, finer-grained distinctions, such as “infotainment” and “docudrama” that is blurring the lines between information proper and the fantasy-part that are being mixed and which ought to result in rough or fuzzy boundaries of the classes, or at least groups of TV programmes that are difficult, if not impossible, to classify automatically.

The third “contemporary cultural heritage” (???) with AI, or at least close to society, is football/soccer, and the seeming folly of robocup – the football cup for robots – which provides “a standard problem where wide range of technologies can be integrated and examined”. Moreover, “RoboCup chose to use soccer game as a central topic of research, aiming at innovations to be applied for socially significant problems and industries” (copied from the Robocup website). As the invited speaker Manuela Veloso explained with great enthusiasm and conviction, it’s not about soccer but, in her specialisation, modeling and implementing game theoretic team strategies. That is, how to represent team strategies, how to generate a team response to an adversary, and how to make strategic decisions in timed zero-sum games? A first step is to separate out skills (actions) from tactics (sequence of actions) from plays (web of tactics). Another aspect is the notion of winning: the usual game-theoretic maximum reward pay-off versus a threshold-win scenario (see also [2], which has won the Outstanding Paper Award at AAAI’07). One contradictory issue in the presentation, however, was the team-individual (non-)balance. To win Robocup, Veloso et al use (a.o.) team strategies alike the “playbooks” in American football that specify multiple play-plans. During “locker-room agreements”, the coach decides which play plan to execute (which can be changed during the game), and all players stick to these rules; hence, the individual player is subordinate to the team. In contrast to this American football, with football/soccer, one regularly sees games with individuals who not always play as a team. Veloso is passionate about the approach off the team-oriented playbook approach of American football, but at the end of the presentation the take-home message for future research was that the team-approach was wrong and that one should look for player strategies that are based on the individual and who only cooperate when “needed”… which pretty much ends up as the average football/soccer game of which she laments the play strategy. To put it positively, I think she probably meant that it is the balance between individual behaviour of the player/robot and the team of players/robots as a whole; hence, when, how, and why switch between these two fundamentally different strategies.

Manuela Veloso and Daniele Nardi during the invited talk (picture from AI*IA website).

[1] Maurizio Montagnuolo and Alberto Messina. TV Genre Classification Using Multimodal Information and Multilayer Perceptrons. Proc. of AI*IA ’07, Rome, 2007.
[2] Colin McMillen and Manuela Veloso. Thresholded Rewards: Acting Optimally in Timed, Zero-Sum Games. In Proceedings of AAAI’07, Vancouver, Canada, July 2007.