Author Archives: Andrés

Semantics revisited

We have seen many “topics” or subdomains becoming more or less popular in the wide range of things considered “artificial intelligence” in the last decades. Even the term artificial intelligence became a bit of a term to avoid when presenting research proposals twenty years ago. Neural networks have a long history and yet ten years ago most people were not hearing about them.

Now we see a new come back: that of semantic technologies. We used to talk a lot about semantics in the nineties and in the early part of this century. Still, the concrete approaches did not became scalable. Other approaches to data management took hold.

Still, slowly but surely, semantics started to appear again, even if not so well understood by many who were supposed to apply it. In the last few years people in the NLP and related domains started to use the word semantics in the context of word embeddings and document embeddings. Open data became more important and suddenly more people started to realise ontologies can be based or enriched by machine learning approaches.

Some of the more exciting things I have seen out there:

  • the improvement of graph data bases such as Neo4J and Amazon Neptune (itself connected to other interesting services)
  • the spread of Query and to a lesser extent Gremlin
  • the appearance of some tools such as Owlready for interaction with the Python sphere
  • the maturity of resources such as DBPedia, Wikipedia and a pletora of projects connected to these
  • the improvement of algorithms for graph manipulation

There are lots of interesting challenges that we need to tackle now in order to solve real life problems. One of them is how to optimise versioning and automatic growth of ontologies with external resources, how to protect personal data in these systems and how to represent ever more complexed relations, specially those explaining “stories”.

I believe this is where we need to explore at a very abstract and then concrete level what I will call high order syntax. Linguists have worked for millenia on syntactic problems. Software specialists have worked on the syntax of programming languages for several decades now. Likewise ontology experts have been developing ever more complex frameworks to express n-nary relationships. Now we need semantic theories that help us tell and manipulate stories with data. And then we will need to spread the knowledge.

Currently there are lots of people working in the NLP spectrum who do not have an understanding of syntax in the linguistic sense and they also lack knowledge about syntax in the sense of semantic languages such as OWL. They talk about “language models” when they are experimenting with parameters for optimising transformers for this or that recognition of text and images. Computational linguists and semantic specialists are needed in order to develop more comprehensive frameworks so that digital systems can somehow tell or recognise stories and, more importantly, react upon them in a reliable way.

I recommend two newish books for those interested in semantic technologies: Knowledge Graphs, Fundamentals, Techniques and Applications, by Keyriwal et alia (2021) and, a little bit more mundane but still interesting, Ontologies with Python by Lamy Jean-Baptiste (also 2021).

Readings of last year

I did not blog for a while. Last year I read a few books that I found productive, inspiring. On the professional side I read mostly blog posts and specific sites on machine learning. On the general side I found these books particularly compelling:

How we learn by Stanislas Dehaene. This is the second book I read by this French neurologist. The first one was about the brain and mathematics. In this book Dehaene goes more into how learning processes happen in the brain and he does some useful comparison between our neural networks and the very primitive networks used in machine learning.

Gates of Europe by Harvard professor Serhii Plokhy presents a comprehensive history of Ukraine. This is really fascinating reading for those interested in the whole debate about history and politics of the Poland-Ukraine-Russia-Belarus region.

Tussen drie plagen by Jaan Kross: I have not finished this yet, it is over 1200 pages long, three books in one. Jaan Kross was a famous Estonian writer. He wrote this tetralogy between 1970 and 1980. It deals with the life of Balthasar Russow and the Livonian war. I am reading the Dutch translation of the book, which got a prize for the best translation from Estonian into another language in 2020. It is an amazing book also for those interested in that corner of the world. Trivial but not so trivial fact: back then you could travel in winter by sleighs pulled by horses from Tallinn to what later became Helsinkin. Nowadays that part of the Baltic Sea does not freeze like that at all. You learn a lot about the relations with the rising empire of the Muscovites, with the Swedes, with the Germans and also a bit about other Slavic groups living in areas constantly fought for between Germans and Poles.

Jaan Krooss in 1938 Image

Another view at history and the present

Early this year I read Frankopan’s Silk Road. The book tries to give a different view of world history with a focus on Asia, in particular central Asia. I liked the book, especially as I was interested in the history of Central Asia since I was about 10 years old.

I think even for a history buff the book contains a lot of fascinating details, from details about slavery in Europe to the way the Vikings did trade across Eurasia to the Sogdians.

There were a couple of points I found weak:

1- some of the regions or countries Frankopan sees as emerging might have been “emerging” mostly because they have been oil and gas exporting countries and we know the times of oil and gas might have already be over. His view on development in Russia and the countries of the Middle East is a bit too optimistic.

2- although Frankopan does well in indicating how many times the West plundered other regions he does not seem to be judging the same kind of processes when they were carried out by the others.

I am still waiting for a book that is going to cover in a more comprehensible way the expansion of China.

Still, the book is a good start for talking about the East, especially now in the times of the coronavirus, when China is booming and Europe and North America are even more dependent on everything coming from there.

Generative Deep Learning

David Forster’s book Generative Deep Learning offers a general view on generative methods for several domains in ML. This is not a book for state-of-the-art recipes on generative DL for the domains you are already working into. The field is evolving very fast and one will surely want to check out the usual data science blogs and above all play with the most popular Github projects in one’s domains. But if one wants to have a general view of how generative DL is used in other ML domains and one likes sound mathematical explanations, this is a very good book.

I come from an NLP world and I became curious about the parts of this book talking about painting and composing. As on my free time I love drawing, I got a very nice introduction here on how we can let machines draw a bit. The book lets you above all think about how to work on solutions combining generative ML for different domains.

Indo-Dutch fiction

One of the last fiction books I read last year was Alfred Birney‘s De Tolk van Java. As far as I know the book has only appeared in Dutch – I am writing this 1 January 2020- but an English translation will be available in a few months and the translation will be The Interpreter of Java. It won the Libris prize for best Dutch novel in 2017 and there are good reasons for that: it has a compelling story, the character description is impeccable, there is a wealth of interesting information on the history and culture of Indonesia and the Indonesian community in the Netherlands and the use of the Dutch language is superb.

Dutch army fighting back pro-Independence Indonesians (Wikipedia)

The book is about the complex relation between an “Indo”, a half-Indonesian, half-Dutch man, and his children, in particular one of them. Chapters switch from a narrative told by the father and those told by one of his children. The father was born in Java of a Chinese-Indonesian mother and an Indonesian-born Dutchman. Chapters switch from his life during the Japanese occupation of Indonesia to that in the Netherlands, back to Japanese occupation and the war of Independence. Things keep going back and forth between those sections and the traumatic experience his children and wife had with him in the Netherlands.

For me it was also interesting to connect some of the facts and impressions I learnt about the Japanese occupation of Indonesia with those I read four years ago in Richard Flanagan’s The Narrow Road to the Deep North. Styles and characters are different but both books helped me to puzzle together a bit of the history of a region we do not know much about in the Americas or Europe. Both books are fiction but the historical backgrounds they portray seem to be very accurate, as far as I have been able to judge by delving into a few reference sources here and there. There are many layers in De Tolk van Java: racism in both the Netherlands and Indonesia, identity, ethnicity, family violence, love.

The Big Challenges and Musk

Musk is arguably one of the most controversial figures in technology nowadays. He has undoubtedly contributed to several interesting technological trends that go beyond the “let’s focus on optimizing clicks” kind of trend we have had in the last couple of decades. At the same a lot of criticism has been raised about the way he does business and how human resources are treated in his companies.

The Milky Way as we still can see in some places (source: Wikipedia)

What I want to start talking here is about his work in space and how this stands against the interests of humans in general. Musk is a big fan of XX century science fiction and a lot of it revolves on the idea that the world as we know is going to come to an end soon and we need to save humanity by conquering space.

OK, that is very cool but there are several issues that appear: how much of our energies do we need to spend in space exploration now as compared to dealing with much less sexy topics as getting grips with rubbish generation worldwide or having a plan once automation speeds up a couple of factors of what we have now?

One little issue now: how are the skies going to look in 10 years from now if all the satellites Musk wants to get around the Earth get there? What will happen if Starlink keeps doing what it is doing? You can read here a bit about what scientists are discussing now. The sad part of this is that the vast majority of people in the nations producing this technology do not have a clue about how the Milky Way looks once you get out of their highly urbanized areas. They do not know what we are going to lose there and they do not even know how much even the very polluted skies we have in Europe or urban North America will look if Musk and others keep sending satellites around. What is the cost? What is the gain? What are the alternatives?

I think scientists need to explain in more visual ways what we are about to lose.

A bit more on the brain

One of the latest books I read on the brain was Hjernen er stjernen by Norwegian neurologist Kaja Nordengen. The book came up in Norwegian back in 2016. The literal translation is “The Brain is a Star” but it got an English translation as “Your Brain is a Superstar”. I find both titles a bit corny but the book is a very pleasant reading. It has a lighter style than Swaab’s books I mentioned some months back and it is rather short. There will surely be a few things laymen interested in neurology would know – never mind neurologists- but most, including me, will learn a few things. I am still not sure about what Nordengen wrote on some specific food items and brain health but I will keep on reading on this very topic. At the very least she woke my interest on the relationship between neuron activity and food (beyond what we read about nuts, omega 3 and the like).

Instagram and Python

There is this article in ZDNet about how Instagram, a heavy user of Python, is trying to push for some processes to tame the Python code mess. I find their ideas quite sensible. I am still amazed at how a programming language has taken so much time to develop best practices and mechanisms for big production time.

If you compare it with Java, which is a bit “younger”, the contrast is huge.

Another thing I have been thinking lately is: why do we have so little to show when it comes to version control and general deployment of machine learning models?

Let’s hope there is more movement in the next couple of years.

I

French fiction

I had not read French literature in months and I finally got to read Au revoir là-haut by Pierre Lemaitre. It was an excellent book: there is a good plot and, above all, language and character descriptions are superb.

It also helps you understand a bit of France at the end of World War I and what came shortly afterwards.

Here you can watch a trailer to the film, which came up in 2017.

Slapen: waarom?

Ik heb zojuist “Why We Sleep”van Matthew Walker afgelezen. Dat is een van de beste non-fiction boeken die ik in twee jaar gelezen heb.

Aan de ene kant slapen

Ik had al lang geleden een uitstekende populairwettenschappelijke boek over het thema gehad, Counting Sheep van David Martin. Dat was interessante mengeling van biologische en culturelle, vooral literaire aspecten van slapen en dromen. Walkers boek brengt nieuwe aspecten met zich, het focus is eerder de gezondheid van de mensen, zelfs als het ook in het boek van Martin eerover ging.

Hier een paar van de thema’s in dit boek:

Hoe werkt precies – zeer precies – cafeïne en welke uitwirkingen die eigenlijk heeft

Wat is eigenlijk de nucleus suprachiasmaticus

Hoe functioneert het circadiaan ritme en melatonine

wat is concreet slapen, hoe fonctionnert de REM-slaap en hoe de non-REM-slaap, o.a wat doen onze neuronale netwerken erbij precies

wat doen andere dieren en hoezo kwam het ertoe

wat doen de mensen specieel die andere dieren niet doen bij het slapen

hoe verandert zich slapen met tijd – spoiler: oudere mensen hebben zoveel slaap nodig als iemand die 20 jaar oud is-

slapen en creativiteit

slapen en opmerkzaamheid

en dutje doen of niet

kanker, slaap en levensverwachting

overgewicht, DNA en slapen

dromen en nut, zeer concreet voor REM en non-REM

slaap en ziekten van het slapen

hoe de maatschapij tegen slapen ging, slapmiddelen en veel meer

slapen, werken, artsen, ziekenhuizen

een nieuwe vizie voor slapen

Ik kan het boek van harte aanbevelen. Ik heb vele puncten hier en daar gelezen maar niet zo in detail en zo goed uitgelegd.

En hier een goede video: