receiving sysex and storing it in ram

Home Forums General Programming receiving sysex and storing it in ram

Viewing 20 posts - 1 through 20 (of 36 total)
  • Author
    Posts
  • #72499
    EnzoF04
    Participant
      • Topics: 17
      • Replies: 102
      • Total: 119
      • ★★

      When i ask my sampler to do a sample dump via midi (sysex) it sends blocks of data. How can i catch this data and store it in ram so i can work / modify it?

      I need to acknowledge the data i’ve received. Every package is 120 bytes long and closed with a checksum in the 121st byte.

      basic question is how do I store the bytes that i requested from my sampler. Sometimes it can be as big as 20000 bytes.

      #72505
      Possemo
      Participant
        • Topics: 14
        • Replies: 638
        • Total: 652
        • ★★★

        You have to do that with the built in Lua script language. If you know how to code or want to learn it I can give you further information.

        • This reply was modified 6 years, 8 months ago by Possemo.
        #72507
        EnzoF04
        Participant
          • Topics: 17
          • Replies: 102
          • Total: 119
          • ★★

          OK, that’s a good advice! I’m just a newbie so i don’t even know where to code this. I can find the LUA script editor but it has this tree on the left side of the window. Where to start??
          Thanks for helping out!

          #72508
          Puppeteer
          Participant
            • Topics: 16
            • Replies: 185
            • Total: 201
            • ★★

            Easiest way to start is make a control and in the panel on the right, add a method to onValueChange

            Open that method and start coding.

            For receiving SYSEX you’ll need a method that is called on receiving data. All the data checks and launching other methods will need to come from this receive method.

            The Puppeteer
            http://godlike.com.au

            #72512
            Possemo
            Participant
              • Topics: 14
              • Replies: 638
              • Total: 652
              • ★★★

              There are demo panels from Roman in the Ctrlr folder but I learned a lot from Carl Licroy’s panels such as the attached JX-8P panel. Look in the Lua editor for a method called midiMessageReceived. This method is attached to Ctrlr’s midi receive slot (see attached screenshot). That’s how you catch Midi data with Lua in Ctrlr.

              Attachments:
              You must be logged in to view attached files.
              #72515
              EnzoF04
              Participant
                • Topics: 17
                • Replies: 102
                • Total: 119
                • ★★

                Very useful replies, guys! I really appreciate this! I’ll work on this tonight and report back!

                • This reply was modified 6 years, 8 months ago by EnzoF04.
                #72546
                EnzoF04
                Participant
                  • Topics: 17
                  • Replies: 102
                  • Total: 119
                  • ★★

                  OK so I can send the request for information to my midi device. The device is sending data back to the panel. I can’t figure out how to store it but I THINK it is writen into an array. I can’t write the contents of this array to a UI element for now. That comes later.

                  When my panel asks for data via sysex the device sends me 20 bytes. I need to acknowledge the received 19 bytes so the device continues sending the next part of sysex bytes.

                  When my atari ask for the sysex info from the device i see a message sent to the device, a request for info. Then i see the atari sending ACKnowledge sysex messages to the device. (All coming “From port 1”).

                  The device is sending a sysex message that builds up as the ACKnowledge messages are set by the atari.

                  How do i build up this awnsering of acknowledge messages while storing the incomming sysex data into an array?

                  second attachement is is the correct sysex message, when i send a request (first message in attachement 1) from my panel i only receive a 20 bytes message.

                  • This reply was modified 6 years, 8 months ago by EnzoF04.
                  Attachments:
                  You must be logged in to view attached files.
                  #72551
                  EnzoF04
                  Participant
                    • Topics: 17
                    • Replies: 102
                    • Total: 119
                    • ★★

                    I think i need someting like
                    on midi message receive
                    put bytes into an array of undefined size
                    when 19 bytes are received (In a certain format)
                    send acknowledge to device
                    –then starts data block 0–
                    when 120 bytes are received calculate checksum
                    if checksum is OK
                    send acknowledge to device
                    if checksum is NOT OK
                    send NOT acknowledged, request retransmission of block
                    if byte received is F7 (EOX)
                    transmission is complete

                    I don’t know how to efficiently code this into lua. I’m affraid timing will be crucial to get the bi-directional communication exactly working.

                    #72553
                    goodweather
                    Participant
                      • Topics: 45
                      • Replies: 550
                      • Total: 595
                      • ★★★

                      Hi, I would proceed as described above by defining a MidiMessageReceived method that you set to the panel property “On Midi Message received”.
                      In that method you check the message size and based on it you perform your actions.
                      So, when receiving 19 bytes, you could check for some byte and if ok send your ACK and so on…

                      Something like:

                      MidiMessageReceived = function(MidiMessage)
                      	
                      	-- Called when the panel receives a Midi message
                      	-- console ("MIDI message received...")
                      
                      	-- Find the Midi message type based on its size
                      	s = MidiMessage:getSize()
                      
                      	-- 13	bytes: Device inquiry reply
                      	-- 27 	bytes: Global Parameters Data dump
                      	-- 1176 bytes: Program Edit Buffer Dump
                      	-- 1178	bytes: Program dump
                      
                      	if s == 14 then
                      --		console ("Device inquiry reply recognized")
                      		if MidiMessage:getData():getByte(6) == 0x2C then
                      			bPro2Ready = true
                      		else
                      			bPro2Ready = false
                      		end
                      
                      	elseif s == 1176 then
                      ...
                      	elseif s == 20 then
                      	else
                      	end
                      end

                      You can look at my Pro2 panel… Good luck!

                      #72556
                      EnzoF04
                      Participant
                        • Topics: 17
                        • Replies: 102
                        • Total: 119
                        • ★★

                        OK that helps! No i can receive a sysex message and pull info from the bytes. I use this code:

                        
                        function midiMessageReceived(MidiMessage)
                        	s = MidiMessage:getSize()
                        	msgAck = CtrlrMidiMessage({0xf0, 0x7e, 0x7f, 0xf7})
                        
                        	if MidiMessage:getData():getByte(3) == 0x00 then
                        		console ("Sample Parameters for sample 01 received")
                        	elseif MidiMessage:getData():getByte(3) == 0x01 then
                        		console ("Sample Parameters for sample 02 received")
                        	elseif MidiMessage:getData():getByte(3) == 0x02 then
                        		console ("Sample Parameters for sample 03 received")
                        	elseif MidiMessage:getData():getByte(3) == 0x03 then
                        		console ("Sample Parameters for sample 04 received")
                        	elseif MidiMessage:getData():getByte(3) == 0x04 then
                        		console ("Sample Parameters for sample 05 received")
                        	elseif MidiMessage:getData():getByte(3) == 0x05 then
                        		console ("Sample Parameters for sample 06 received")
                        	else
                        		console ("message not recornized")
                        	end
                        	if MidiMessage:getData():getByte(18) ==0x00 then
                        		panel:sendMidiMessageNow(msgAck)
                        		console ("Acknowledge message sent")
                        	end
                        end
                        

                        This works for now! I now need to analyze the received midi message on bytes 6,7,8 (Sample period) and bytes 9,10,11 (total Words in sample).
                        Thanks so far!

                        #72557
                        EnzoF04
                        Participant
                          • Topics: 17
                          • Replies: 102
                          • Total: 119
                          • ★★

                          OK So i’ve received this sysex message.
                          (in decimal)

                          240 126 001 000 000 012 049 099  007 072 001 000 072 001 000 072
                          001 000 000 247

                          (in hexadecimal)

                          f0 7e 01 00 00 0c 31 63  07 48 01 00 48 01 00 48
                          01 00 00 f7

                          I know bytes 9,10,11 represent the total words in sample. How do i convert decimal 007 072 001 (decimal) or 07 48 01 (in hexadecimal) to words.

                          #72558
                          goodweather
                          Participant
                            • Topics: 45
                            • Replies: 550
                            • Total: 595
                            • ★★★

                            Do you have example of what a “word” is?
                            Easiest is to take a sound that you know the values from the synth; do a dump and analyze it…

                            #72559
                            EnzoF04
                            Participant
                              • Topics: 17
                              • Replies: 102
                              • Total: 119
                              • ★★

                              f0 7e 01 00 00 0c 31 63 07 48 01 00 48 01 00 48 01 00 00 f7
                              byte 5 (0c) = bits per word = 012
                              bytes 6,7,8 (31 63 07) = sampling period in nS

                              
                              HEX  DEC
                               31  049 lsb  (    1 x 049 =    049)
                               63  099      (  128 x 099 =  12672)
                               07  007 msb  (16834 x 007 = 114688)
                              049+12672+114688 = 127409
                              

                              bytes 9,10,11 (48,01,00) = total words in sample

                              
                              HEX  DEC
                               48  072 lsb  (    1 x 072 = 072)
                               01  001      (  128 x 001 = 128)
                               00  000 msb  (16834 x 000 = 000)
                              072+128+000 = 200 words in sample
                              

                              bytes 12,13,14 (48,01,00) = Loop start point

                              
                              HEX  DEC
                               48  072 lsb  (    1 x 072 = 072)
                               01  001      (  128 x 001 = 128)
                               00  000 msb  (16834 x 000 = 000)
                              072+128+000 = 200 words in sample
                              

                              bytes 15,16,17 (48,01,00) = Loop End Point

                              
                              HEX  DEC
                               48  072 lsb  (    1 x 072 = 072)
                               01  001      (  128 x 001 = 128)
                               00  000 msb  (16834 x 000 = 000)
                              072+128+000 = 200 words in sample
                              

                              byte 18 (00) = Scan type, 0=looping, 0=alternating (one shot if loop length <5).

                              #72561
                              EnzoF04
                              Participant
                                • Topics: 17
                                • Replies: 102
                                • Total: 119
                                • ★★

                                so i want to compute this but it won’t work…

                                 	wordPart01 = MidiMessage:getData():getByte(9)
                                		console (string.format (wordPart01))
                                	wordPart02 = MidiMessage:getData():getByte(10)
                                		console (string.format (wordPart02))
                                	wordPart03 = MidiMessage:getData():getByte(11)
                                		console (string.format (wordPart03))
                                 	wordPartLes = (tonumber(wordPart01) * 1)
                                 	wordPartMiS = (tonumber(wordPart02) * 128)
                                 	wordPartMoS = (tonumber(wordPart03) * 16384)
                                 	wordCount = ((wordPartLes + wordPartMis) + wordPartMos)
                                 		console (string.format(wordCount))
                                #72565
                                goodweather
                                Participant
                                  • Topics: 45
                                  • Replies: 550
                                  • Total: 595
                                  • ★★★

                                  Ha ok! Good that I read that you know what a word is (in your sampler terms) and that you can compute in hexa (I mean assembling bytes).
                                  So, it’s only a Lua problem…
                                  – to display a byte in the console you can use the tostring() function as
                                  console(tostring(MidiMessage:getData():getByte(10)))
                                  Doing so you get the decimal value
                                  – do you get the correct 3 values?
                                  – MidiMessage:getData() is a Memory block as such that you can extract bytes/ ranges / copy / replace partly / …
                                  – you can directly add the values
                                  iTotalWords = MidiMessage:getData():getByte(9) + 128 * MidiMessage:getData():getByte(10) + 16384 * MidiMessage:getData():getByte(11)
                                  – then you display it with console(tostring(iTotalWords)) or you use it directly to display in your panel
                                  modLCDline1 = panel:getModulatorByName(“LCDline1”) — I’m doing all my declaration in a separate method read at panel startup to avoid calling getModulatorByName each time which is not good
                                  modLCDline1:getComponent():setPropertyString (“uiLabelText”, tostring(iTotalWords))

                                  Let me know how it goes 😉

                                  #72568
                                  EnzoF04
                                  Participant
                                    • Topics: 17
                                    • Replies: 102
                                    • Total: 119
                                    • ★★

                                    This helps to make the sysex message more readable! Very good explanation, goodweather, this is getting me closer!
                                    – you told that you do all the declarations in a specific part, where and how? Those are global decelerations?

                                    – I am coming to a realization that I need exact timing for sending a message back to the sampler as byte 18 is received in a longer sysex message of 319 bytes. When I do not awnser with a 4 byte acknowledge received data, the sampler sends an EOX, end of exclusive. I’ve tried something like if midiMessage get size < 17 or equal to 18, sendMidiMesage now. Console shows me that the condition is met and a midi monitor app shows sending of the acknowledge sysex message but sampler is not continuing sending the other bytes. It sends an EOX. Thanks for the great advises!

                                    • This reply was modified 6 years, 8 months ago by EnzoF04.
                                    #72573
                                    goodweather
                                    Participant
                                      • Topics: 45
                                      • Replies: 550
                                      • Total: 595
                                      • ★★★

                                      No pbm 🙂

                                      I have created a method called “AssignModulators” that contains all my declarations of modulators variables. You can have a look at my Pro2 panel.

                                      function AssignModulators()
                                      
                                      	-- Main Screen
                                      	lblParameter = panel:getModulatorByName("lblParameter")
                                      	txtInformation = panel:getModulatorByName("txtInformation")
                                      	lblCurrentValue = panel:getModulatorByName("lblCurrentValue")
                                      	lblCurrentValue:getComponent():setPropertyString ("uiLabelText", "Current value")
                                      	txtCurrentValue = panel:getModulatorByName("txtCurrentValue")
                                      	lblSavedValue = panel:getModulatorByName("lblSavedValue")
                                      	txtSavedValue = panel:getModulatorByName("txtSavedValue")
                                      	txtBank = panel:getModulatorByName("txtBank")
                                      	lblProgram = panel:getModulatorByName("lblProgram")
                                      	txtProgram = panel:getModulatorByName("txtProgram")
                                      	txtProgramName = panel:getModulatorByName("txtProgramName")
                                      	modBank = panel:getModulatorByName("Bank")
                                      	modProgram = panel:getModulatorByName("Program")
                                      
                                      	-- Oscillator 1
                                      	modOsc1On = panel:getModulatorByName("Osc1On")
                                      	modOsc1Sync = panel:getModulatorByName("Osc1Sync")
                                      	modOsc1Pitch = panel:getModulatorByName("Osc1Pitch")
                                      	modOsc1FineTune = panel:getModulatorByName("Osc1FineTune")
                                      ...

                                      As stated by atom (creator of Ctrlr), getModulatorByName is looking in a table to find the modulator and in that table there is no order so it is needed to read each name.
                                      Therefore, for efficiency reason it is better to assign all your modulators to variables and then use those variables in all your code.

                                      In Lua, all variables are global by default unless you state them explicitely with local. So pay attention when re-using variable names in different methods/functions. This is great but some care is needed.

                                      For your load in different steps, you need to use a main call initiating the load, a MidiMessageReceived method containing different actions based on size length but also a method containing different timers.
                                      With one Timers method you can handle several timers (by ID). The advantage is that you can tweak the timer time for each section.

                                      So the sequence would be something like:
                                      – push Load button launch Load method sending request to synth
                                      – MidiMessageReceived gets first part and sends second request and starts timer to send next request
                                      – MidiMessageReceived gets next part and sends next request and starts another timer to send next request until finished
                                      – last MidiMessageRceived would do the final processing of the complete message by launching another function “HandleLoadedData()”

                                      Look at my Pro2 panel. I’ll support you further with timers.
                                      Good luck en fijne avond 😉

                                      #72574
                                      EnzoF04
                                      Participant
                                        • Topics: 17
                                        • Replies: 102
                                        • Total: 119
                                        • ★★

                                        I’m getting some results:
                                        analyzing incoming sysex data and displaying it
                                        and another request for a sample with different data received

                                        #72575
                                        EnzoF04
                                        Participant
                                          • Topics: 17
                                          • Replies: 102
                                          • Total: 119
                                          • ★★

                                          Ah I get it (assigning modulators on panel initialisation. In the “calles when panel has finished loading” you put a method called PanelLoaded. In this method you call a function called “AssignModulators”where all the variables are set with their corresponding on panel modulators.

                                          This is clear and for a rebuild very handy. I’m thinking of making stuff more modular instead of coding it all 6 or 16 times (for the samples e.g.).

                                          Further more on the load in different steps. This is quiet difficult, I’m afraid. Because of the fact that I send a request for sample dump to the S700. Then the S700 starts sending sysex bytes. While sending it the S700 needs to receive an ACKS byte after sending the first 19 bytes. See what the implementation documentation is telling about the communication outline. (attachement)
                                          Fijne avond, insgelijks! :*)

                                          Attachments:
                                          You must be logged in to view attached files.
                                          #72577
                                          EnzoF04
                                          Participant
                                            • Topics: 17
                                            • Replies: 102
                                            • Total: 119
                                            • ★★

                                            Is a timer sync’ed to incoming bytes? Let’s say

                                            if MidiMessage:getSize() = 18 then
                                            timer = timer+1
                                            end

                                            I’m doing:

                                            	msgSize = MidiMessage:getSize()
                                            	msgAck = CtrlrMidiMessage({0xf0, 0x7e, 0x7f, 0xf7})
                                            
                                            	if msgSize == 19 then
                                            		panel:sendMidiMessageNow(msgAck)
                                            	end
                                            

                                            But i’m realising that the getSize is completed after the whole sysex msg is received? Am I right?

                                            • This reply was modified 6 years, 8 months ago by EnzoF04.
                                          Viewing 20 posts - 1 through 20 (of 36 total)
                                          • The forum ‘Programming’ is closed to new topics and replies.
                                          There is currently 0 users and 64 guests online
                                          No users are currently active
                                          Forum Statistics
                                          Threads: 2,495, Posts: 17,374, Members: 77,605
                                          Most users ever online was 12 on January 22, 2019 3:47 pm
                                          Ctrlr