Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering... more
Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering deep understanding, improvement, and parallel execution. This paper develops a simple, generalized AD algorithm calculated from a simple, natural specification. The general algorithm is then specialized by varying the representation of derivatives. In particular, applying well-known constructions to a naive representation yields two RAD algorithms that are far simpler than previously known. In contrast to commonly used RAD implementations, the algorithms defined here involve no graphs, tapes, variables, partial derivatives, or mutation. They are inherently parallel-friendly, correct by construction, and usable directly from an existing programming language with no need for new data types or programming style, thanks to use of an AD-agnostic compiler p...
Research Interests:
Research Interests:
Research Interests: Higher order and RTA
Animation is a temporally continuous phenomenon, but is typically programmed in terms of a discrete sequence of changes. The use of discreteness serves to accommodate the machine that is presenting an animation, rather than the person... more
Animation is a temporally continuous phenomenon, but is typically programmed in terms of a discrete sequence of changes. The use of discreteness serves to accommodate the machine that is presenting an animation, rather than the person modeling an animation with the help of a computer. Using a continuous model of time for animation allows for natural specification, avoiding some artificial details, but is difficult to implement with generality, robustness and efficiency. This paper presents and motivates continuous modeled animation, and sketches out a naive functional implementation for it. An examination of some of the practical problems with this implementation leads to several alternate representations, all of which have difficulties in themselves, some quite subtle. We hope that the insights and techniques discussed in this paper lead to still better representations, so that animation may be specified in natural terms without significant loss of performance.
Research Interests:
Research Interests:
Research Interests:
This paper describes a program for allowing a user to navigate smoothly in a 2D (possibly animated) image, performing pan, zoom and rotation. The program is written in Haskell, using the Fran interactive animation library. The idea is... more
This paper describes a program for allowing a user to navigate smoothly in a 2D (possibly animated) image, performing pan, zoom and rotation. The program is written in Haskell, using the Fran interactive animation library. The idea is that when a hand grasps the image, the image stuck to the hand at the grasped location. When only one hand is grasping, the image pans (translates). When two hands are grasping, the image pans, zooms and rotates as required in order to keep the two grasped locations under the hands. While the C++ program that inspired this exercise is quite long and complex, the Fran version is short, modular and we feel captures the essence of the behavior being modeled.
Research Interests:
Research Interests:
Research Interests:
ABSTRACT
Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering... more
Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering deep understanding, improvement, and parallel execution. This paper develops a simple, generalized AD algorithm calculated from a simple, natural specification. The general algorithm is then specialized by varying the representation of derivatives. In particular, applying well-known constructions to a naive representation yields two RAD algorithms that are far simpler than previously known. In contrast to commonly used RAD implementations, the algorithms defined here involve no graphs, tapes, variables, partial derivatives, or mutation. They are inherently parallel-friendly, correct by construction, and usable directly from an existing programming language with no need for new data types or programming style, thanks to use of an AD-agnostic compiler p...
Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering... more
Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering deep understanding, improvement, and parallel execution. This paper develops a simple, generalized AD algorithm calculated from a simple, natural specification. The general algorithm is then specialized by varying the representation of derivatives. In particular, applying well-known constructions to a naive representation yields two RAD algorithms that are far simpler than previously known. In contrast to commonly used RAD implementations, the algorithms defined here involve no graphs, tapes, variables, partial derivatives, or mutation. They are inherently parallel-friendly, correct by construction, and usable directly from an existing programming language with no need for new data types or programming style, thanks to use of an AD-agnostic compiler p...
Research Interests:
Research Interests:
... http://research.microsoft.com/˜conal/papers/ icfp97.ps. [8] Sigbjorn Finne and Simon Peyton Jones. Pictures: A simple structured graphics model. ... http://haskell.org/ definition, February 1999. [14] PeterLucas and Stephen N. Zilles.... more
... http://research.microsoft.com/˜conal/papers/ icfp97.ps. [8] Sigbjorn Finne and Simon Peyton Jones. Pictures: A simple structured graphics model. ... http://haskell.org/ definition, February 1999. [14] PeterLucas and Stephen N. Zilles. Graphics in an applicative context. ...
Research Interests:
We present a user-friendly approach to unifying program creation and execution, based on a notion of “tangible values” (TVs), which are visual and interactive manifestations of pure values, including functions. Programming happens by... more
We present a user-friendly approach to unifying program creation and execution, based on a notion of “tangible values” (TVs), which are visual and interactive manifestations of pure values, including functions. Programming happens by gestural composition of TVs. Our goal is to give end-users the ability to create parameterized, composable content without imposing the usual abstract and linguistic working style of programmers. We hope that such a system will put the essence of programming into the hands of many more people, and in particular people with artistic/visual creative style.
In realizing this vision, we develop algebras for visual presentation and for “deep” function application, where function and argument may both be nested within a structure of tuples, functions, etc. Composition gestures are translated into chains of combinators that act simultaneously on statically typed values and their visualizations.
In realizing this vision, we develop algebras for visual presentation and for “deep” function application, where function and argument may both be nested within a structure of tuples, functions, etc. Composition gestures are translated into chains of combinators that act simultaneously on statically typed values and their visualizations.
Research Interests:
An image manipulation system can be thought of as a domain-specific programming language: by composing and manipulating pictures, the user builds up an expression in such a programming language. Each time the picture is displayed, the... more
An image manipulation system can be thought of as a domain-specific programming language: by composing and manipulating pictures, the user builds up an expression in such a programming language. Each time the picture is displayed, the expression is evaluated. To gain the required efficiency of display, the expression must be optimized and compiled on the fly. This paper introduces a small language that could form the basis of an image manipulation system, and it describes a preliminary implementation. To compile an expression in the language, we inline all function definitions and use algebraic laws of the primitive operations to optimize composites. We apply a form of code motion to recover (as far as possible) the sharing lost in the inlining phase. Finally, we generate intermediate code that is passed to a JIT compiler.
A very general problem-oriented CAI system is described that allows an individual with no programming knowledge to control the nature and content of student-machine interaction. A functioning stand-alone system is outlined for instruction... more
A very general problem-oriented CAI system is described that allows an individual with no programming knowledge to control the nature and content of student-machine interaction. A functioning stand-alone system is outlined for instruction in linear equations of variable complexity.
Research Interests:
Type classes provide a mechanism for varied implementations of standard interfaces. Many of these interfaces are founded in mathematical tradition and so have regularity not only of types but also of properties (laws) that must hold.... more
Type classes provide a mechanism for varied implementations of standard interfaces. Many of these interfaces are founded in mathematical tradition and so have regularity not only of types but also of properties (laws) that must hold. Types and properties give strong guidance to the library implementor, while leaving freedom as well. Some of this remaining freedom is in how the implementation works, and some is in what it accomplishes.
To give additional guidance to the what, without impinging on the how, this paper proposes a principle of type class morphisms (TCMs), which further refines the compositional style of denotational semantics. The TCM idea is simply that the instance’s meaning is the meaning’s instance. This principle determines the meaning of each type class instance, and hence defines correctness of implementation. It also serves to transfer laws about a type’s semantic model, such as the class laws, to hold for the type itself. In some cases, it also provides a systematic guide to implementation, and in some cases, valuable design feedback.
The paper is illustrated with several examples of types, meanings, and morphisms.
To give additional guidance to the what, without impinging on the how, this paper proposes a principle of type class morphisms (TCMs), which further refines the compositional style of denotational semantics. The TCM idea is simply that the instance’s meaning is the meaning’s instance. This principle determines the meaning of each type class instance, and hence defines correctness of implementation. It also serves to transfer laws about a type’s semantic model, such as the class laws, to hold for the type itself. In some cases, it also provides a systematic guide to implementation, and in some cases, valuable design feedback.
The paper is illustrated with several examples of types, meanings, and morphisms.
Research Interests:
This paper describes a program for allowing a user to navigate smoothly in a 2D (possibly animated) image, performing pan, zoom and rotation. The program is written in Haskell, using the Fran interactive animation library. The idea is... more
This paper describes a program for allowing a user to navigate smoothly in a 2D (possibly animated) image, performing pan, zoom and rotation. The program is written in Haskell, using the Fran interactive animation library. The idea is that when a hand grasps the image, the image stuck to the hand at the grasped location. When only one hand is grasping, the image pans (translates). When two hands are grasping, the image pans, zooms and rotates as required in order to keep the two grasped locations under the hands. While the C++ program that inspired this exercise is quite long and complex, the Fran version is short, modular and we feel captures the essence of the behavior being modeled.
Animation is a temporally continuous phenomenon, but is typically programmed in terms of a discrete sequence of changes. The use of discreteness serves to accommodate the machine that is presenting an animation, rather than the person... more
Animation is a temporally continuous phenomenon, but is typically programmed in terms of a discrete sequence of changes. The use of discreteness serves to accommodate the machine that is presenting an animation, rather than the person modeling an animation with the help of a computer. Using a continuous model of time for animation allows for natural specification, avoiding some artificial details, but is difficult to implement with generality, robustness and efficiency. This paper presents and motivates continuous modeled animation , and sketches out a naïve functional implementation for it. An examination of some of the practical problems with this implementation leads to several alternate representations, all of which have difficulties in themselves, some quite subtle. We hope that the insights and techniques discussed in this paper lead to still better representations, so that animation may be specified in natural terms without significant loss of performance. A shorter version of this report appeared in the proceedings of PLILP/ALP '98, and is © Springer-Verlag .
Research Interests:
Research Interests:
... 14. Mark Pesce, VRML Browsing and Building Cyberspace: Browsing and Building Cyberspace, New Riders Publishing, Thousand Oaks, CA, 1995. 15. John Peterson and Gary Shu Ling [1997], "Fran User's... more
... 14. Mark Pesce, VRML Browsing and Building Cyberspace: Browsing and Building Cyberspace, New Riders Publishing, Thousand Oaks, CA, 1995. 15. John Peterson and Gary Shu Ling [1997], "Fran User's Manual", http://www.haskell.org/fran/fran.html. ...
While interactive multimedia animation is a very compelling medium, few people are able to express themselves in it. There are too many low-level det ails that have to do not with the desired content—e.g., shapes, appearance and... more
While interactive multimedia animation is a very compelling medium, few people are able to express themselves in it. There are too many low-level det ails that have to do not with the desired content—e.g., shapes, appearance and behavior—but rather how to ge t a computer to present the content. For instance, behaviors like motion and growth are generally grad ual, continuous phenomena. Moreover, many such behaviors go on simultaneously. Computers, on the other hand, cannot directly accommodate either of t hese basic properties, because they do their work in dis crete steps rather than continuously, and they only do on e thing at a time. Graphics programmers have to spen d much of their effort bridging the gap between what an animation is and how to present it on a computer. We propose that this situation can be improved by a change of language, and present Fran, synthesized by complementing an existing declarative host language , Haskell, with an embedded domain-specific vocabulary for modeled animation. As demonstrated in a collec - tion of examples, the resulting animation descripti ons are not only relatively easy to write, but also hig hly composable.
Research Interests:
... Also supports construction and deconstruction in rectangular and polar/spherical coordinates. 2-D and 3-D transforms. ... Page 5. 5 A Brief Introduction to ActiveVRML Colors. Various constants,construction, and deconstruction in... more
... Also supports construction and deconstruction in rectangular and polar/spherical coordinates. 2-D and 3-D transforms. ... Page 5. 5 A Brief Introduction to ActiveVRML Colors. Various constants,construction, and deconstruction in RGB and HSL color spaces. Text. ...
Research Interests:
Research Interests:
Research Interests:
Research Interests:
... data D a = Daa deriving (Eq, Show) constD :: Num a ⇒ a → D a constD x = D x 0 idD :: Num a ⇒ a → D a idD x = D x 1 instance Num a ⇒ Num (D a) where fromInteger x = constD (fromInteger x) Dxx + Dyy = D (x + y) (x + y ) Dxx ∗ Dyy = D (x... more
... data D a = Daa deriving (Eq, Show) constD :: Num a ⇒ a → D a constD x = D x 0 idD :: Num a ⇒ a → D a idD x = D x 1 instance Num a ⇒ Num (D a) where fromInteger x = constD (fromInteger x) Dxx + Dyy = D (x + y) (x + y ) Dxx ∗ Dyy = D (x ∗ y) (y ∗ x + x ∗ y) negate (Dxx ) = D ...