A Universe of Sorts

§ Transformer Architecture is based on sets, not sequences

Neat text from Francois Chollet
He highlights that the correct way to think of a transformer is as an architecture that attends to a set of tokens.
This makes it more powerful than an RNN, since we do not "bias" in any direction.
It does make for a wrinkle: how do we give it the positions of words in a sentence? Hence, positional encoding.
But really, we can throw in any choice of encoding data we like, to give it "latent information"
In this way, the spatial location is baked into the feature space!

doctrine of the affections: Ornate and flowery writing, producing a specific emotional response. This manifests as long phrases, with fewer cadences (spots where the music rests).
major and minor scales, moving to this tonal system / equal temparement.
In WTC / well tempered clavier, bach showed that he can write in all keys.
dissonant chords are allowed
early baroque: homophonic
late baroque: polyphonic.
terraced dynamics: forte/piano contrasts. Music would be either loud or soft, no inbetween, due to instrumental limitations.
Contrapuntal texture: Lots of little melodies running in and out of each other.
Mostly composed for harpsichord!
music notation has been standardized (since the middle ages)
basso continuo: bassline. improvisation.
polyphonic imitation: another voice enters, singing the same song, overlapping the same melody.
stradivarius made string instruments. Around 1000 were made, 600 exist.

bartolomeo cristofori: inventor of piano
piano was a modification of the harpsichord
harpsichord was the main keyboard instrument, baroque era, strings plucked by quills
Bach: harpsichord No 1 in D minor (beautiful harpsichord piece)
Bach's primary instrument was the organ, and would also compose for the harpsichord.

Sacred Music
Concreto Grosso: 2-4 soloists, 8-20 musicians. Ritornello form, alternation between tutti and solo sessions.
Fugue: polyphonic composition with imitation voices. Main melody in a fugue is called the subject. Melodic lines are called voices, and are layered.
Fugue has Inversion, Retrograde, Retrograde Inversion, Augmentation, Diminuition.
Bach, Little fugue in G minor.
Mnemonic to remember that bach came first: "Bach laid the track, Beethoven brought it back". Also,
"Mozart’s melodies moved after Bach’s Baroque brilliance bloomed"

Perfect Pitch
Photographic memory: Wrote down a piece that was never released by the sistine chapel by just hearing it.
Child prodigy: touring as a child.

Mnemonic to remember that bach came first: "Bach laid the track, Beethoven brought it back".
Unhappy childhood. Father was alcoholic, had to support his mother by performing.
Played piano for mozart, was invited to vienna at age 17. He was helped by the aristocracy.
Perfect pitch, lost hearing later in life.
Bridge to the romantics.
Moonlight sonata.

Video
Mozart "sweet", beethoven "fiery".
Mozart had "beauty" and "perfection" in his composition. (functional programmer!)
Mozart composed for the harpsichord, beethoven for piano.
In the sonata, mozart in the development section (after the exposition section), is much more controlled than beethoven. Beethoven goes farther texturally.
My personal take: mozart has a "stacatto" feel, while beethoven has a "legato" feel.

He was fascinated by music boxes and clocks.
Bolero: features a simple, repetitive theme that is introduced by a snare drum and gradually layered with different instruments, creating an ever-increasing intensity.