Category Archives: Uncategorized

Natural Language Processing with TensorFlow by Ganegadara

I actually started to read this book last year. I went through most of it and experimented a lot but had no time to write. I finally read the last chapter, which I had somehow put off. All in all, it is a useful book on using TensorFlow for NLP.

It offers a good exploration of what Word2Vec is. It goes on to CNNs first with image recognition and then with sentence classification, then it does a good initial cover of LSTMS and finally it touches very briefly some trends.

I think I would have skipped the NLP introduction, but then I have worked on NLP almost all of my life. Ganegadara should not have gone so long on WordNet etc. As much as it was an usual tool, it still might be, a paragraph would have been enough.

It was a pity the last part, the one I left until now on trends, was so short but then the technology and even the basics on what deep learning is about are changing so fast!

Another beautiful mind

Now I am currently reading “A Mind for Numbers: How to Excel at Math and Science” by Barbara Oakley. It is a magnificent book. I am just finishing the third chapter and I know it is one of the best books I have read in the last 12 months. She basically gives lots of insights about learning processes with a focus on mathematics and science but basically for everything. She quotes a few of the authors I have appreciated before like Terrence Deacon. She dispels several myths. I have read quite a few books on neuroscience – for laymen and speciaists – and some on learning in general. There are everywhere good pieces of advise, useful reminders of things most of us know, some new approaches. She is yet another one of those authors and yet she describes the importance of some of them is very good: ways for switching between a focused and an unfocused mode of thinking, for instance, or about dealing with the Einstellung effect.

I am just starting and I am very happy.

Where is C++ heading?

Is it just me or the items that have been discussed for C++17 are the lamest innovations in many years of C++ development? What with the removals of trigraphs and the like?

Well, there is still some time for 2017…sigh. Let’s see what the meetings at the end of this year bring.

 

Panthera_leo_-zoo_-yawning-8a

A good general book about machine learning

Peter Flach wrote a very neat introduction to machine  learning. The ISBN is 978-1107422223. It is more on a theoretical side than on the practical, but it also contains a lot of good pieces of advice about concrete decisions professionals have to make on selection of features, testing and so on. Mathematical points are clearly explained. I usually don’t mention this but the layout is also very pleasant, with lots of charts helping to visualize the issues discussed.

I would have liked to read more on kernel methods. A bit more of detail on the algorithmic part would also have been welcome. Still, it is a good reference book.

 

Nuestros relojes internos

En agosto compré un libro sobre los relojes biológicos. El título en alemán es ” Wie wir ticken: Die Bedeutung der Chronobiologie für uns” (Cómo funciona nuestro reloj: la importancia de la cronlogía para nosotros). El científico que lo escribió, Till Roelleberg, es un alemán, pero el libro es una versión de una publicación en inglés. En la Red se pueden ver algunos vídeos donde este autor explica algunas de sus ideas en inglés y alemán.

Circadian_rhythm_labeled

Hay un montón de datos interesantes sobre cómo funcionan los mecanismos de regulación del tiempo en humanos y otros seres vivo.

Si ven el artículo sobre cronobiología en la Wikidia alemana, podrán comprobar cómo este tema parece haber despertado más interés en la comunidad alemana.

Entre otras cosas, uno lee algo sobre el funcionamiento del núcleo supraquiasmático. Hubiera preferido que el autor hubiese dedicado un capítulo entero a este tema, pero algo es algo.

También hay datos interesantes sobre los tipos de cronogramas que cada persona tiene. Realmente somos poco conscientes de cómo funcionan nuestros relojes biológicos y de cómo nuestras convenciones sociales afectan de manera tan dramática nuestros procesos fisiológicos porque van contra nuestros cronómetros internos, que pueden ser muy diversos.

Siempre me habían interesado cuestiones referentes al jet lag, a la importacia del dormir bien, a la influencia del sol, pero ahora estoy un poco más pendiente de estos asuntos. Piensen un momento: la hora en que sale y se oculta el sol influye a todos los mamíferos, incluídos los humanos. Aun así, en Madrid, París y Berlín se tiene el mismo huso horario. Hoy en Berlín el sol salió a las 7:40 am mientras que en París salió a las 8:14. Muchas personas en ambos sitios se levantan exactamente  al mismo tiempo y van al trabajo al mismo tiempo.

Lamentablemente, el estilo del libro me molestó bastante. El autor comienza cada capítulo con una historia inventada. Pone nombres a sus personajes, pero también a científicos imaginarios. Luego, en la segunda parte del capítulo, explica que tal o cual científico imaginario se llama en realidad de tal o cual manera. Eso distrae. Bien habría podido referirse a los investigadores reales. Aparte de eso, hay cosas repetitivas. Aun así, el libro es una introducción interesante a este tema.

 

Computing a mind

Some time ago I finished Ray Kurzweil’s “How to create a mind, the secret for human thought”. The book is definitely worth reading, also for someone working in any sub domain of artificial intelligence.

We are beyond this, surely

We are beyond this, surely

Kurzweil recapitulates his work on artificial intelligence and relates it to the latest research on how minds – biological minds, to be more specific – work. I found funny he goes into some of the work Nuance is carrying  out with medicine and semantics. I worked on some items from a company called Language & Computing which Nuance acquired several years ago.

One of the chapters I liked the most was chapter 4, The Biological Cortex. I have read a few popularizing books on neurology and a few more technical articles about neurological research, but there were a few new items to me here. Chapter 5 is also pretty fascinating for me. Obviously, the neurological part is the one where I am an absolute layman. By all means this makes me want to read more about it.

Kurzweil states the basic units of the neocortex are modules composed of around one hundred neuros. According to him – and some other researchers – connections within each module are mostly inherited, they are based on our genes, whereas connections between modules are above all the product of experience, of training.

Chapter 7, the Biological Inspired Neocortex is the one with the most things where I wanted to gloss over, but I still read it through. Here I started to find more things I have my second thoughts about. Firstly, Kurzweil claims somehow that although “no human can claim to have mastered all of Wikipedia…[it] is only part of Watson’s knowledge base. Is it? I do not thin so. I have worked a lot with Wikipedia and have studied a zillion projects trying to garner as much new semantic information as possible in the most reliable way (a lot of these projects providing data for DBPedia, but a lot more to private companies) and I wouldn’t say we are even close to producing a “full” extraction of Wikipedia information. How would it even look like? Even if we are talking about a purely statistical analysis of Wikipedia pages: what does that mean? Statistics are, one way or the other, attached to a model, to a hypothesis. How many hypotheses can we create from any middle-size page of Wikipedia?

Kuzweil warns about how difficult the concept of consciousness is and how even the most rational scientists can get into the vaguest rumblings here. And yet, for me he fails to deliver here. He gets as fuzzy as anyone else. In that chapter 7 he goes over the Turing test and makes his predictions about when a computer will pass the Turing test. He discusses Searle’s Chinese room and argues that if if what Searle said about that room is true, “the human brain would not be judged capable of thinking either”.

Perhaps the issue I have is that I don’t agree even with what the Turing test is supposed to mean. We recently saw how the University of Reading competition went. When I read the dialogues that made a third of the judges believe they were dealing with a human being, I was surprised, to put it mildly, that anyone could fall for it. Now, we need to ask: what is the difference between the 1/3 who believed the bot was a human and the 2/3 who didn’t? I don’t think the answer is “they are less clever”.  At least I think that cannot be the only explanation. I reckon the 1/3 of gullible people know less about what we know today of what kind of reactions we can automate.

I think consciousness, at least as we see it in humans, is very related to a very deep degree of self-reference, the existence of a system that is aware of itself, of itself being aware of itself and and so on recursively. If we relate this system to the idea of the Chinese room: human consciousness would be capable not only of acting as a Chinese room towards the external world, but of creating internal, virtual Chinese rooms it can analyse on itself and in itself, virtual Chinese rooms and other kinds of rooms it can think of.

All in all, although Kurzweil tends to exaggerate here and there, his writing is readable, he provide some interesting pieces of information for everyone and he contributes to the discussion about how we can build more intelligent computer systems.