Transforming Magic Paper Magic has long been associated with the mystical and the unexplained, captivating the imaginations of people throughout history. From ancient times to the present day, the concept of magic continues to intrigue and inspire. One fascinating aspect of magic is the ability to transform objects into something completely different. This concept is beautifully exemplified by the idea of transforming magic paper. Imagine a piece of ordinary paper that, with the wave of a wand or a simple incantation, can be transformed into a myriad of extraordinary forms and shapes. The transformative power of magic paper knows no bounds.
Step 14: Pick up all 8 boxes and turn over to the other side.
Moreover, we equip our model with effective depth-wise separable convolutions to learn the specific feature representations and trade off computational complexity. The first question is whether such a visual medium can be used to do serious mathematical exploration, the kind a researcher might do when working on a research problem.
The transformative power of magic paper knows no bounds. It can be folded into intricate origami creations, with delicate cranes taking flight or intricate flowers blooming before your eyes. The paper can be stretched and pulled, becoming a dazzling display of geometric patterns and illusions.
Magic Paper
Working notes by Michael Nielsen, November 2017. Followup to (but doesn't require) my notes on Chalktalk.
[Technology] transforms what can be said and what is said… all major advances in consciousness depend on technological transformations in implementations of the word. – Walter Ong, Interfaces of the Word (1977).
Mathematical ideas are conventionally expressed using notation and terminology developed using static media. Suppose, however, that mathematics had been invented after modern computers. This is perhaps difficult to imagine – after all, mathematics helped lead to computers – but let's do the thought experiment anyway. Might mathematical notation have developed in a different way? Would we instead have developed a dynamic, interactive notation more powerful than the static mathematical and linguistic notations in common use today?
As part of my exploration of this question, I recently developed a simple prototype system, which I call Magic Paper (code). To get a feel for what the system does, below is a video of me using Magic Paper to explain the (very simple!) basic idea underlying the proof of a beautiful theorem about artificial neural networks: the result that neural nets may be used to approximate any continuous function* * See Approximation by superpositions of sigmoidal function, by George Cybenko (1989), and Multilayer feedforward networks are universal approximators, by Kurt Hornik, Maxwell Stinchcombe, and Halbert White (1989). . Note that you do not need to be deeply familiar with neural networks to follow along; an introduction to the basics of neural nets may be found here. You may wish to fullscreen the video, for easier viewing:
The design of Magic Paper was inspired in part by a system developed by Ken Perlin, known as Chalktalk. Chalktalk has not been publicly released at the time of this writing (November 2017)* * Update: Chalktalk has since been released, see here. , but you can get some understanding of the system by viewing from 32:40 to the end of the following video:
Can a visual medium be used to do serious mathematical exploration?
I built Magic Paper to explore two questions not directly addressed by Chalktalk.
The first question is whether such a visual medium can be used to do serious mathematical exploration, the kind a researcher might do when working on a research problem. In other words: can such a system be used as a medium for creative thought? While Chalktalk is a fun medium to explore in, it's not obvious to me that it's suitable for the kind of exploration that arises in mathematical work.
Of course, it's too ambitious for such a quickly-built prototype medium to be useable across all of mathematics. And so I decided to narrow the question, and ask whether such a medium could be used to explore at least one non-trivial mathematical question; concretely: can neural nets be used to approximate any function?
While focusing on one problem is much easier than building a general-purpose exploratory mathematical medium, it's still an extremely demanding and thus informative constraint. Furthermore, it seems likely that patterns useful in solving one mathematical problem will be reuseable in solving others.
In fact, even this is too demanding. The idea for a proof presented above wasn't actually discovered in this medium. It's a proof taken from my book on neural networks* * Michael A. Nielsen, Neural Networks and Deep Learning (Chapter 4), Determination Press (2015). . Despite this, I believe this context – explaining the idea underlying the proof of a relatively recent mathematical result – is a good one for understanding what is required for a general-purpose exploratory mathematical medium.
Note that while the video of Magic Paper perhaps makes it appear to be explanatory or educational in intent, that is not the case. It is intended primarily to be a medium to think and explore in, and only secondarily to explain or to educate in. While these goals are sometimes congruent, they are also often in tension.
As an example of this tension, explanatory media are often highly polished, presenting a finished product from which distracting elements have been eliminated. By contrast, exploration involves distractions, wrong paths, and dead ends. One sees this in myriad small ways in the video: the frequent readjustment of the position and size of the glyphs; the various unnecessary weight and bias parameters that are sometimes shown; and so on. In a polished presentation these would have been eliminated, showing just what the viewer needs at any given moment. But such superfluous elements are inevitable in exploration.
Another example of the tension between education and exploration comes from the programming language Logo. Logo is often touted as an example of discovery-based learning, and in many respects it's a good exploratory medium for mathematics. However, while Logo has had considerable success in helping children understand ideas about differential geometry, differential geometers do not, as far as I'm aware, use it as a medium for research. That's potentially a problem: when education and real practice are separated, it can result in busy work on poorly-motivated “educational” problems. An interesting project would be to create a single environment which can be used for both education and research. I am optimistic this is possible; after all, beginning chess players use the same board and the same moves as the world champion.
Can we reify deep ideas about mathematics inside the Magic Paper interface?
The second question motivating Magic Paper was whether it is possible to reify deep ideas about mathematics inside the interface. That sounds cryptic, but has a straightforward meaning. To explain that meaning, let's briefly switch topic, and recall an idea from elementary physics, the principle of conservation of energy:
½mv 2 + U(x) = constant
The basic point is that there's some relationship between the mass of a particle, m, its velocity v, its position x, and its potential energy U(x). The way physics is ordinarily done, we manipulate this expression using algebraic operations. But a different approach is possible. In my essay “Thought as a Technology”* * Michael Nielsen, Thought as a Technology, available at http://cognitivemedium.com/tat/index.html (2016). I develop a prototype interface which reifies the principle of conservation of energy. In particular, the prototype vividly shows the user the relationship between mass, position, velocity and potential energy. This makes it possible to directly see and adjust the potential energy, see how the trajectories respond, and so on. This gives a unique and rich experience, one enabling the user to internalize a powerful set of cognitive technologies for thinking about motion.
Chalktalk-like media seem promising for reifying such principles. Unfortunately, as far as I'm aware there are no general results about neural networks as powerful as conservation of energy. However, as a step in this direction, directly seeing the relationship between the parameters of a network and the output of various neurons does help us understand the behaviour of the network. Further work could incorporate deeper ideas about neural networks, such as learning by gradient descent, or regularization, or ways of visualizing the network* * A nice recent example is the paper by Jason Yosinksi, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson, on Understanding Neural Networks Through Deep Visualization ICML (2015). . It'd also be interesting to develop such an interface in a context where more deep principles are available – perhaps using Magic Paper to explore (say) chaotic dynamical systems, using ideas such as conservation of energy, conservation of phase space volume, Lyapunov exponents, and the KAM theorem. Building such ideas into the interface would make them a routine, concrete part of the way the user works and thinks.
Why not just key commands?
Instead of drawing a neuron, why not just hit “n”? Instead of drawing axes, why not just hit “g”, for “graph”? Wouldn't such keystrokes be faster and easier?
This question has bothered me ever since first seeing Perlin demonstrate Chalktalk. Even as an audience member, I found it much more satisfying to see him draw a glyph than to just hit a key and have it appear. And this effect was even stronger as a user of Magic Paper.
I don't have a satisfactory understanding of why it's satisfying to do this. Perhaps it's because the act of drawing activates associations in my brain that wouldn't have been activated as strongly by issuing a key command. In this sense, the act of drawing is far more meaningful – and meaning-inducing – than hitting a key. It's a little like the reasons we don't name a variable meant to represent someone's name as jnwhg; it's more meaningful to use name. Or the reasons it's better to use a map of a coastline to navigate, rather than a log of latitudes and longitudes for points on the coast.
Unfortunately, while such explanations are suggestive they're vague and incomplete. And yet my lived experience is that this is a strong effect. Something similar is true of physical activities, such as tennis or dance: there is an enormous difference between a symbolic representation – say, reading about tennis – versus actually doing it. Now, for the most part, we tend to think of these activities as a special class of “physical” activities, different from “mental” activities such as mathematics. Yet I'm not sure the distinction is all that sharp.
More broadly, advocates of embodied cognition often make strong claims about its importance. As far as I can see, the evidence they have for those claims is weak. And yet while the specific claims may be weak or wrong, I believe there is some strong and important underlying effect. It really does greatly help my thinking to sketch the glyphs in Magic Paper. I just don't understand why.
Working within the system
I began the design work for Magic Paper with paper-and-pen storyboards and a text-based script for my video. As I developed the storyboards and script I also began to develop the system. Very quickly, the system was powerful enough that I could do basic sketches. Once that point was reached, it was more stimulating (and easier) to work within the new medium, and to abandon the storyboards and script.
I initially felt guilty at abandoning the storyboards and script, as though that was somehow the “correct” way of working. But working in the new medium quickly began to suggest useful ideas that I do not believe I would otherwise have anticipated.
As an example, in the final video we often see multiple graphs showing the behaviour of several different neurons in a network. This makes it easy for the viewer to understand the behaviours of different parts of the network, and to put it all together:
By contrast, my initial storyboards and script had just a single output graph. It was only after I began using the medium that I realized it was possible to use the graphs in this way. In particular, I didn't design the system so it could plot the behavior of intermediate neurons. Rather, that was a fortuitous happenstance, tried spontaneously as I used the medium.
Another point I did not anticipate in storyboarding and scripting was how useful it would be to move and resize the neurons and the graphs, e.g.:
A big benefit of this is object persistence. When possible, it's often useful to reuse or repurpose existing objects, since viewers have already built strong mental models of their behaviour. This kind of dynamic reuse is harder to achieve in static media. Perhaps for that reason I did not anticipate it in my storyboarding and scripting.
By contrast, my earlier description of this proof in an online book consisted of text interspersed with interactive elements, e.g.:
Many of the interactive elements depict minor variations on the same system. But because there is no object persistence, the user must parse each one separately, and put in the mental work required to understand the relationships. For the most part this is not terribly hard work, but it is an additional mental burden.
All told, while the script and storyboards were useful to bootstrap, once the medium was running, it was more generative to work directly within that, experimenting and evolving an approach I was comfortable with.
Extensions: abstracting glyphs, re-parameterization, glyph design
Here are three extensions that could be added to Magic Paper with 40 to 50 hours of additional work.
First, it should be possible to “lasso” several elements, combining them into a single new glyph. For instance, we could lasso the two-neuron gadget used to build tower functions, and turn it into a single re-usable glyph. That abstracted glyph could then be duplicated, moved, its parameters edited, and so on, just as any other glyph. It would also be possible to inspect the internal state of the abstracted glyph, making it possible to understand multiple layers of abstraction.
Second, it should be possible to re-parameterize the abstracted glyph, so it has just three parameters, representing the height of the tower function, a starting x value, and a finishing x value. These three parameters would then determine the biases and weights of the underlying elements. This could be implemented with a simple expression parser.
Third, it should be possible to integrate an image editor enabling the addition of extra glyphs to Magic Paper's visual language. Amongst other things, such an editor could be used to add unique visual representations for the abstracted glyphs.
Conclusion
My research colleagues sometimes express puzzlement at my interest in new media forms. They often treat it as fun but not especially serious work. To them, advanced research usually means work on esoteric subjects. But people working on new media forms have the opportunity to rethink absolutely fundamental concepts such as number, equality, and so on; such work is about rethinking what mathematics and language is. I believe such experiments will eventually affect and perhaps transform the way all research is done. But it will take a huge effort, since this kind of rethinking is not an individual-sized problem, it is a civilization-sized problem, and we're just getting started.
With the right spell, it can even become a floating lantern or a flying bird, defying the laws of physics and leaving spectators in awe. But the true beauty of transforming magic paper lies not just in its physical transformations, but in the emotional and imaginative impact it has on those who witness its magic. Seeing a plain sheet of paper come to life before your eyes invokes a sense of wonder and childlike curiosity. It reminds us that even the simplest objects can hold extraordinary potential and that magic exists in the everyday world. Moreover, transforming magic paper can also serve as a metaphor for personal growth and transformation. Just as the paper undergoes a metamorphosis, so too can individuals change and evolve. It inspires us to embrace our own potential for transformation and encourages us to see the magic within ourselves. In conclusion, transforming magic paper is a captivating concept that showcases the enchanting powers of magic. From its physical transformations to its emotional impact, it serves as a reminder of the wonder and possibility that exist in the world. So, the next time you come across a plain piece of paper, imagine the limitless potential it holds and consider the magic that could be unleashed with just a flick of your wrist..
Reviews for "From puzzles to interactive games: Transforming magic paper in the digital age"
1. Sarah - 1 star - I was really excited to try out the transforming magic paper, but I was extremely disappointed with the results. The paper claimed to magically change colors when water was added, but it barely changed color at all! I followed the instructions perfectly, but the paper only had a faint hint of color. It was nowhere near the vibrant transformation I was expecting. Save your money and skip this product!
2. David - 2 stars - The transforming magic paper didn't live up to its hype for me. The concept sounded intriguing, but the execution fell flat. The color change was very subtle and hardly noticeable, even with multiple attempts. Additionally, the paper started to tear easily when wet, leading to further frustration. Overall, it wasn't worth the price and I wouldn't recommend it.
3. Emily - 1 star - I wish I could give the transforming magic paper zero stars. It was a complete waste of money. The paper didn't transform at all! I followed the instructions carefully, adding water slowly, but there was no change in color whatsoever. I even tried a different batch of paper, thinking I received a defective one, but the result was the same. It's misleading advertising to claim that this paper can transform when it clearly does not work as claimed.
4. Michael - 2 stars - While the transforming magic paper did change color slightly when water was added, it was far from impressive and not worth the price. The color change was faint and the paper itself felt flimsy and low quality. It also took a relatively long time to dry, making it inconvenient for any quick art projects. Overall, it was a letdown and I would not purchase it again.