Coloured notes

Synthesia is a living project. You can help by sharing your ideas.
Search the forum before posting your idea. :D

No explicit, hateful, or hurtful language. Nothing illegal.
Post Reply
spaceman
Posts: 2

Post by spaceman »

I have experienced learning songs using coloured notes and it really speeds up and improves memory enormously.
I would love to see this is Synthesia.
Nicholas
Posts: 13135

Post by Nicholas »

We'll be able to add this feature once MS Squared's patent on this expires around the year 2023. (They recently took legal action against Sibelius for including the very same thing.)
spaceman
Posts: 2

Post by spaceman »

That is complete madness.

Are they going to sue people with synesthesia as well?
Or are they just going after the big companies with money :roll:

The US patent system is a sad joke.
Nicholas
Posts: 13135

Post by Nicholas »

spaceman wrote:That is complete madness.
I agree.

I'm not really sure what the "obviousness test" covers anymore. In the ruling I linked last time, the judge even has a footnote on the last page that says: "The Court also notes that patents appear to have been issued for more than a century protecting systems of musical notation, including systems based on assigning colors to tones. See, e.g., US Patent No. 122096 (filed Dec 26, 1871)...". I guess having the same idea 130 years later still doesn't make it obvious? :?

It's a shame, too. Per-pitch color would be a useful visualization.
User avatar
jimhenry
Posts: 1899

Post by jimhenry »

The proceedings against Sibelius are at an early stage. There has not been any sort of hearing before the court yet. Sibelius moved for a judgment in the case before trial on the basis that the patent claims an unpatentable abstract idea. The judge denied the motion on the basis that the question of whether or not patent eligible subject matter was claimed could not be decided at this early stage. Based on a quick skim of the claims, it does look like the claims are unpatentable as being claims to abstract ideas.

In any event, it also seems like the claims are intended to be directed to associating colors in a related sequence with notes sequentially around the circle of fifths so that harmonic relationships are apparent from the colors. Therefore, it would seem that most arrangements for applying colors to notes would not be covered by the MS2 patent. Certainly you could use the color scheme covered by the 122,096 patent with impunity.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
Nicholas
Posts: 13135

Post by Nicholas »

Yeah, there is a long Google Groups topic in which the author of that patent describes his thoughts on the circle of fifths, here. He even mentions specifically: "Once again spectrum color cannot be related to the chromatic octave but can only be related to the circle of 5ths. Correlating color to the chromatic octave introduces abstractions in that the meaning of the associated colors do not correspond to what is being perceived aurally."

I guess if you have a patent on something, it's probably standard operating procedure to disclaim the benefits of other methods. :lol:

If I locked it down to specifically that color scheme in the 122,096 patent, you're right that it would be safe. The scary part in the original Avid complaint was that an Avid employee mentioned in a forum post how to customize the colors, which apparently "induces infringement" of the MS2 patent.
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

Does the patent cover following scenario, too:

1. Synthesia allows 12 colors, color1..color12 for assigning to each pitchclass. So nothing is preassigned.
2. Users themselves do the assigning work. Synthesia will allow any RGB mix.
Nicholas
Posts: 13135

Post by Nicholas »

That exact scenario is mentioned in the complaint against AVID's Sibelius. Once Sibelius realized they were infringing on the patent, they removed the automatic colors and left them user-assigned. But by saying "now you guys can set the colors yourself", the Sandborn case asserts they were now "inducing infringement" instead of directly infringing.

(I know, it's silly.)
TonE
Synthesia Donor
Posts: 1180

Post by TonE »

I do not know, I am not an expert, but would not listen to any experts, too. Those experts seem to have often 'chicken mentality'. Synthesia, by default, could assign transparent color setting, with the user having the ability to change it, if the country in which they live allows it. Why excluding countries from this possibility where is no patent protection? Russia? North Korea?
LikeARollingStone
Posts: 15

Post by LikeARollingStone »

TonE wrote:I do not know, I am not an expert, but would not listen to any experts, too. Those experts seem to have often 'chicken mentality'. Synthesia, by default, could assign transparent color setting, with the user having the ability to change it, if the country in which they live allows it. Why excluding countries from this possibility where is no patent protection? Russia? North Korea?
That is a good idea, and I agree it's unlikely that the patent holder could have legal rights to deny people to choose a color for their notes. It would almost be as mad as denying people to use Indian ink on their piano keys.

I've tried getting in touch with the patent holder to see what software uses his «system», but his mailbox is full, and here we have another company with other colors that probably don't give him a dime

http://m.youtube.com/watch?v=gdpq1av2pfQ
LikeARollingStone
Posts: 15

Post by LikeARollingStone »

I tried buying stickers with colors from the patent holder, to attach to a piano/keyboard, but they weren't being sold separately anymore. Seems like the person behind the patent makes his living by tutoring via Skype, and is using the patent as a roadblock rather than as a cash cow.
LikeARollingStone
Posts: 15

Post by LikeARollingStone »

This seems like it was outdated already when it was being made.
http://www.virtuosoism.com/new.htm

I think what he is doing is unethical if he indeed makes it impossible for others to use the colors ascribing, and at the same time isn't more active than he is.
tomislav
Posts: 4

Post by tomislav »

Nicholas wrote:We'll be able to add this feature once MS Squared's patent on this expires around the year 2023. (They recently took legal action against Sibelius for including the very same thing.)
Can't we just allow people to use any color on any note they want (via .xml config file)?
That cannot break any patent. And anyone can then chose their own color. That way Synthesia is legally safe.
Nicholas
Posts: 13135

Post by Nicholas »

tomislav wrote:That cannot break any patent.
Well, if you meant changing each note's color one at a time, that is probably safe enough.

If you meant changing each pitch's color ("All D's should be green"), that's actually what the legal action I linked in the last message was about. Sibelius just allowed users to change colors per pitch, and the holder of that patent sued them because it could be used to infringe his patent. Although, I agree it's totally ridiculous.
suti
Posts: 2

Post by suti »

why dont you colour the background of the note instead of the note itself? Wait I will patent that :roll:
tomislav
Posts: 4

Post by tomislav »

Ok, I understand the predicament.

Does that mean you actually will not offer support for this feature until the case gets resolved (i.e. 2013) ?

Anyway this is ridiculous.
For instance, what if I hack Synthesia and do it myself. Would you (Synthesia dev team) still be eligible as a legal target?
What if ...
Oh, I am sure you have tons of ideas yourself. Since it is aggravating me more and more I think about it, that's a sign I need to stop doing it :)

Thanks for a great product...
User avatar
jimhenry
Posts: 1899

Post by jimhenry »

Nicholas wrote:If you meant changing each pitch's color ("All D's should be green"), that's actually what the legal action I linked in the last message was about. Sibelius just allowed users to change colors per pitch, and the holder of that patent sued them because it could be used to infringe his patent. Although, I agree it's totally ridiculous.
At the heart of the Sandborn patent US6,930,235 is a scheme for selecting the colors applied to the notes so that they make harmonic relationships more apparent, at least in the view of the inventor.

Sibelius initially offered a feature called "Pitch Spectrum" that colored the notes that allegedly selected the colors in the way claimed in the patent. Then Sibelius removed the feature because of the allegation of patent infringement. Finally Sibelius put in a feature that allowed notes to be colored as specified by the user. Based on this, Sibelius is being sued for past infringement based on "Pitch Spectrum" as originally included. And induced infringement based on providing the program feature needed to infringe AND some form of inducement to use the program to infringe.

Inducing infringement is a tricky concept. Generally the inducer has to do something fairly explicit to induce infringement. Drawing on what I recall as real facts, DDT was known as a pesticide. Someone realized that DDT could also be used for tanning leather. They got a patent on a method for using DDT to tan leather. (That would be called a new use patent.) You could sell DDT without getting in trouble because there is a substantial non-infringing use for DDT, use as a pesticide. But if you sold DDT with instructions for how to use it to tan leather according to the method covered by the patent, you probably would be liable for inducing infringement.

Getting back to colored notes, I don't know what is alleged as the inducement to infringe. It could just be that the old version of Sibelius showed the arrangement of colors alleged to infringe and that is alleged to be inducement to use the new coloring feature to replicate the feature that was removed. It could be that an Avid employee posted explicit instructions on how to use the new feature to do exactly what "Pitch Spectrum" did. The explicit instructions would be pretty good support for inducement to infringe, assuming "Pitch Spectrum" infringed. Only providing the mechanism to replicate a previously infringing feature without some sort of instructions or suggestion to use the new feature to infringe is a bit more iffy. Reading the Complaint, I suspect an Avid employee did post some thoughts on their Forum that to some degree suggested using the new feature to replicate the removed "Pitch Spectrum."

Given the litigious propensities demonstrated by Sandborn, Nicholas' decision to give colored notes a wide berth is understandable. I suspect the economics would have to be something like adding a colored note feature would have to sell an extra 100,000 copies to be worth the risk. Anyone think that is likely?

As already mentioned, adding colors as explained in the 122,096 patent is safe. But that only provides a diatonic (7 note) scale. Nonetheless, it wouldn't be too hard to come up with 12 fixed colors that did not use the relationship covered by the Sandborn patent. Is that what people want? Or do you really want the ability to pick your own colors including the Sandborn type arrangements?
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
tomislav
Posts: 4

Post by tomislav »

As already mentioned, adding colors as explained in the 122,096 patent is safe. But that only provides a diatonic (7 note) scale. Nonetheless, it wouldn't be too hard to come up with 12 fixed colors that did not use the relationship covered by the Sandborn patent. Is that what people want? Or do you really want the ability to pick your own colors including the Sandborn type arrangements?
I could not care less about Sandborn's arrangement. And if Synthesia would add an exception to forbid that exact combination, I don't see how anyone can complain.
But yes, I would love to experiment with colors per octave or individually via some config file. Also, if the same would be applied to staff notes (not only falling notes/bubbles), that would be superb.
(At one point, when reading through open source projects of midi apps, I started replacing staff notes with funny colored images. That was super fun for my son)

This could result in interesting arrangements and maybe even produce new learning techniques. In my experience, people (i.e. users) show remarkable creativity when the developers allow them to express themselves through great software. So far, Synthesia is already a great product, so I dare not create any pressure around this point.

Anyway, thanks jimhenry for your input.
IvanL
Posts: 42

Post by IvanL »

Has anyone ever heard of a case where it was good that a patent existed? I've only ever heard mad stories like this very one. By now, the whole idea of patents sounds primitive to me. "Ooga booga, I make sharp stone quick. I have patent! You make sharp stone slow! If see you make sharp stone quick, I smash!"
User avatar
jimhenry
Posts: 1899

Post by jimhenry »

Just to restate a frequently stated class of "good patents," pharmaceutical companies claim that they could not justify the large costs and risks associated with developing new drugs if it were not for the patent protection they obtain on the drugs. And I know of a few small inventors who have been able to establish successful companies built around a patented product. I know of one such company that almost went out of business in the year following the expiration of its patent.

While one hears horror stories about patents on non-inventions, given the huge numbers of patents issued, some clinkers are bound to get through. But it is not as easy to get a patent as one might think. Almost half of all patent applications do not result in a patent being issued. And there are plenty of cases where something that is inventive does not get a patent because the inventor doesn't have the resources to fight a recalcitrant patent examiner.

Just like the legal system as a whole, there are plenty of examples of the patent subsystem producing unjust results. But, on the whole, having patents is better than not having patents. One indication that a country's economy is maturing is that the country starts paying attention to having a good system for intellectual property protection.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ
http://www.Miditzer.org/
Post Reply