Home Forums General Programming receiving sysex and storing it in ram

This topic contains 35 replies, has 5 voices, and was last updated by  EnzoF04 4 months, 1 week ago.

Viewing 20 posts - 1 through 20 (of 36 total)
  • Author
    Posts
  • #72499

    EnzoF04
    Participant

    When i ask my sampler to do a sample dump via midi (sysex) it sends blocks of data. How can i catch this data and store it in ram so i can work / modify it?

    I need to acknowledge the data i’ve received. Every package is 120 bytes long and closed with a checksum in the 121st byte.

    basic question is how do I store the bytes that i requested from my sampler. Sometimes it can be as big as 20000 bytes.

    #72505
    Possemo
    Possemo
    Participant

    You have to do that with the built in Lua script language. If you know how to code or want to learn it I can give you further information.

    • This reply was modified 4 months, 3 weeks ago by Possemo Possemo.
    #72507

    EnzoF04
    Participant

    OK, that’s a good advice! I’m just a newbie so i don’t even know where to code this. I can find the LUA script editor but it has this tree on the left side of the window. Where to start??
    Thanks for helping out!

    #72508
    Puppeteer
    Puppeteer
    Participant

    Easiest way to start is make a control and in the panel on the right, add a method to onValueChange

    Open that method and start coding.

    For receiving SYSEX you’ll need a method that is called on receiving data. All the data checks and launching other methods will need to come from this receive method.

    The Puppeteer
    http://godlike.com.au

    #72512
    Possemo
    Possemo
    Participant

    There are demo panels from Roman in the Ctrlr folder but I learned a lot from Carl Licroy’s panels such as the attached JX-8P panel. Look in the Lua editor for a method called midiMessageReceived. This method is attached to Ctrlr’s midi receive slot (see attached screenshot). That’s how you catch Midi data with Lua in Ctrlr.

    Attachments:
    You must be logged in to view attached files.
    #72515

    EnzoF04
    Participant

    Very useful replies, guys! I really appreciate this! I’ll work on this tonight and report back!

    • This reply was modified 4 months, 2 weeks ago by  EnzoF04.
    #72546

    EnzoF04
    Participant

    OK so I can send the request for information to my midi device. The device is sending data back to the panel. I can’t figure out how to store it but I THINK it is writen into an array. I can’t write the contents of this array to a UI element for now. That comes later.

    When my panel asks for data via sysex the device sends me 20 bytes. I need to acknowledge the received 19 bytes so the device continues sending the next part of sysex bytes.

    When my atari ask for the sysex info from the device i see a message sent to the device, a request for info. Then i see the atari sending ACKnowledge sysex messages to the device. (All coming “From port 1”).

    The device is sending a sysex message that builds up as the ACKnowledge messages are set by the atari.

    How do i build up this awnsering of acknowledge messages while storing the incomming sysex data into an array?

    second attachement is is the correct sysex message, when i send a request (first message in attachement 1) from my panel i only receive a 20 bytes message.

    • This reply was modified 4 months, 2 weeks ago by  EnzoF04.
    Attachments:
    You must be logged in to view attached files.
    #72551

    EnzoF04
    Participant

    I think i need someting like
    on midi message receive
    put bytes into an array of undefined size
    when 19 bytes are received (In a certain format)
    send acknowledge to device
    –then starts data block 0–
    when 120 bytes are received calculate checksum
    if checksum is OK
    send acknowledge to device
    if checksum is NOT OK
    send NOT acknowledged, request retransmission of block
    if byte received is F7 (EOX)
    transmission is complete

    I don’t know how to efficiently code this into lua. I’m affraid timing will be crucial to get the bi-directional communication exactly working.

    #72553
    goodweather
    goodweather
    Participant

    Hi, I would proceed as described above by defining a MidiMessageReceived method that you set to the panel property “On Midi Message received”.
    In that method you check the message size and based on it you perform your actions.
    So, when receiving 19 bytes, you could check for some byte and if ok send your ACK and so on…

    Something like:

    MidiMessageReceived = function(MidiMessage)
    	
    	-- Called when the panel receives a Midi message
    	-- console ("MIDI message received...")
    
    	-- Find the Midi message type based on its size
    	s = MidiMessage:getSize()
    
    	-- 13	bytes: Device inquiry reply
    	-- 27 	bytes: Global Parameters Data dump
    	-- 1176 bytes: Program Edit Buffer Dump
    	-- 1178	bytes: Program dump
    
    	if s == 14 then
    --		console ("Device inquiry reply recognized")
    		if MidiMessage:getData():getByte(6) == 0x2C then
    			bPro2Ready = true
    		else
    			bPro2Ready = false
    		end
    
    	elseif s == 1176 then
    ...
    	elseif s == 20 then
    	else
    	end
    end

    You can look at my Pro2 panel… Good luck!

    #72556

    EnzoF04
    Participant

    OK that helps! No i can receive a sysex message and pull info from the bytes. I use this code:

    
    function midiMessageReceived(MidiMessage)
    	s = MidiMessage:getSize()
    	msgAck = CtrlrMidiMessage({0xf0, 0x7e, 0x7f, 0xf7})
    
    	if MidiMessage:getData():getByte(3) == 0x00 then
    		console ("Sample Parameters for sample 01 received")
    	elseif MidiMessage:getData():getByte(3) == 0x01 then
    		console ("Sample Parameters for sample 02 received")
    	elseif MidiMessage:getData():getByte(3) == 0x02 then
    		console ("Sample Parameters for sample 03 received")
    	elseif MidiMessage:getData():getByte(3) == 0x03 then
    		console ("Sample Parameters for sample 04 received")
    	elseif MidiMessage:getData():getByte(3) == 0x04 then
    		console ("Sample Parameters for sample 05 received")
    	elseif MidiMessage:getData():getByte(3) == 0x05 then
    		console ("Sample Parameters for sample 06 received")
    	else
    		console ("message not recornized")
    	end
    	if MidiMessage:getData():getByte(18) ==0x00 then
    		panel:sendMidiMessageNow(msgAck)
    		console ("Acknowledge message sent")
    	end
    end
    

    This works for now! I now need to analyze the received midi message on bytes 6,7,8 (Sample period) and bytes 9,10,11 (total Words in sample).
    Thanks so far!

    #72557

    EnzoF04
    Participant

    OK So i’ve received this sysex message.
    (in decimal)

    240 126 001 000 000 012 049 099  007 072 001 000 072 001 000 072
    001 000 000 247

    (in hexadecimal)

    f0 7e 01 00 00 0c 31 63  07 48 01 00 48 01 00 48
    01 00 00 f7

    I know bytes 9,10,11 represent the total words in sample. How do i convert decimal 007 072 001 (decimal) or 07 48 01 (in hexadecimal) to words.

    #72558
    goodweather
    goodweather
    Participant

    Do you have example of what a “word” is?
    Easiest is to take a sound that you know the values from the synth; do a dump and analyze it…

    #72559

    EnzoF04
    Participant

    f0 7e 01 00 00 0c 31 63 07 48 01 00 48 01 00 48 01 00 00 f7
    byte 5 (0c) = bits per word = 012
    bytes 6,7,8 (31 63 07) = sampling period in nS

    
    HEX  DEC
     31  049 lsb  (    1 x 049 =    049)
     63  099      (  128 x 099 =  12672)
     07  007 msb  (16834 x 007 = 114688)
    049+12672+114688 = 127409
    

    bytes 9,10,11 (48,01,00) = total words in sample

    
    HEX  DEC
     48  072 lsb  (    1 x 072 = 072)
     01  001      (  128 x 001 = 128)
     00  000 msb  (16834 x 000 = 000)
    072+128+000 = 200 words in sample
    

    bytes 12,13,14 (48,01,00) = Loop start point

    
    HEX  DEC
     48  072 lsb  (    1 x 072 = 072)
     01  001      (  128 x 001 = 128)
     00  000 msb  (16834 x 000 = 000)
    072+128+000 = 200 words in sample
    

    bytes 15,16,17 (48,01,00) = Loop End Point

    
    HEX  DEC
     48  072 lsb  (    1 x 072 = 072)
     01  001      (  128 x 001 = 128)
     00  000 msb  (16834 x 000 = 000)
    072+128+000 = 200 words in sample
    

    byte 18 (00) = Scan type, 0=looping, 0=alternating (one shot if loop length <5).

    #72561

    EnzoF04
    Participant

    so i want to compute this but it won’t work…

     	wordPart01 = MidiMessage:getData():getByte(9)
    		console (string.format (wordPart01))
    	wordPart02 = MidiMessage:getData():getByte(10)
    		console (string.format (wordPart02))
    	wordPart03 = MidiMessage:getData():getByte(11)
    		console (string.format (wordPart03))
     	wordPartLes = (tonumber(wordPart01) * 1)
     	wordPartMiS = (tonumber(wordPart02) * 128)
     	wordPartMoS = (tonumber(wordPart03) * 16384)
     	wordCount = ((wordPartLes + wordPartMis) + wordPartMos)
     		console (string.format(wordCount))
    #72565
    goodweather
    goodweather
    Participant

    Ha ok! Good that I read that you know what a word is (in your sampler terms) and that you can compute in hexa (I mean assembling bytes).
    So, it’s only a Lua problem…
    – to display a byte in the console you can use the tostring() function as
    console(tostring(MidiMessage:getData():getByte(10)))
    Doing so you get the decimal value
    – do you get the correct 3 values?
    – MidiMessage:getData() is a Memory block as such that you can extract bytes/ ranges / copy / replace partly / …
    – you can directly add the values
    iTotalWords = MidiMessage:getData():getByte(9) + 128 * MidiMessage:getData():getByte(10) + 16384 * MidiMessage:getData():getByte(11)
    – then you display it with console(tostring(iTotalWords)) or you use it directly to display in your panel
    modLCDline1 = panel:getModulatorByName(“LCDline1”) — I’m doing all my declaration in a separate method read at panel startup to avoid calling getModulatorByName each time which is not good
    modLCDline1:getComponent():setPropertyString (“uiLabelText”, tostring(iTotalWords))

    Let me know how it goes 😉

    #72568

    EnzoF04
    Participant

    This helps to make the sysex message more readable! Very good explanation, goodweather, this is getting me closer!
    – you told that you do all the declarations in a specific part, where and how? Those are global decelerations?

    – I am coming to a realization that I need exact timing for sending a message back to the sampler as byte 18 is received in a longer sysex message of 319 bytes. When I do not awnser with a 4 byte acknowledge received data, the sampler sends an EOX, end of exclusive. I’ve tried something like if midiMessage get size < 17 or equal to 18, sendMidiMesage now. Console shows me that the condition is met and a midi monitor app shows sending of the acknowledge sysex message but sampler is not continuing sending the other bytes. It sends an EOX. Thanks for the great advises!

    • This reply was modified 4 months, 1 week ago by  EnzoF04.
    #72573
    goodweather
    goodweather
    Participant

    No pbm 🙂

    I have created a method called “AssignModulators” that contains all my declarations of modulators variables. You can have a look at my Pro2 panel.

    function AssignModulators()
    
    	-- Main Screen
    	lblParameter = panel:getModulatorByName("lblParameter")
    	txtInformation = panel:getModulatorByName("txtInformation")
    	lblCurrentValue = panel:getModulatorByName("lblCurrentValue")
    	lblCurrentValue:getComponent():setPropertyString ("uiLabelText", "Current value")
    	txtCurrentValue = panel:getModulatorByName("txtCurrentValue")
    	lblSavedValue = panel:getModulatorByName("lblSavedValue")
    	txtSavedValue = panel:getModulatorByName("txtSavedValue")
    	txtBank = panel:getModulatorByName("txtBank")
    	lblProgram = panel:getModulatorByName("lblProgram")
    	txtProgram = panel:getModulatorByName("txtProgram")
    	txtProgramName = panel:getModulatorByName("txtProgramName")
    	modBank = panel:getModulatorByName("Bank")
    	modProgram = panel:getModulatorByName("Program")
    
    	-- Oscillator 1
    	modOsc1On = panel:getModulatorByName("Osc1On")
    	modOsc1Sync = panel:getModulatorByName("Osc1Sync")
    	modOsc1Pitch = panel:getModulatorByName("Osc1Pitch")
    	modOsc1FineTune = panel:getModulatorByName("Osc1FineTune")
    ...

    As stated by atom (creator of Ctrlr), getModulatorByName is looking in a table to find the modulator and in that table there is no order so it is needed to read each name.
    Therefore, for efficiency reason it is better to assign all your modulators to variables and then use those variables in all your code.

    In Lua, all variables are global by default unless you state them explicitely with local. So pay attention when re-using variable names in different methods/functions. This is great but some care is needed.

    For your load in different steps, you need to use a main call initiating the load, a MidiMessageReceived method containing different actions based on size length but also a method containing different timers.
    With one Timers method you can handle several timers (by ID). The advantage is that you can tweak the timer time for each section.

    So the sequence would be something like:
    – push Load button launch Load method sending request to synth
    – MidiMessageReceived gets first part and sends second request and starts timer to send next request
    – MidiMessageReceived gets next part and sends next request and starts another timer to send next request until finished
    – last MidiMessageRceived would do the final processing of the complete message by launching another function “HandleLoadedData()”

    Look at my Pro2 panel. I’ll support you further with timers.
    Good luck en fijne avond 😉

    #72574

    EnzoF04
    Participant

    I’m getting some results:
    analyzing incoming sysex data and displaying it
    and another request for a sample with different data received

    #72575

    EnzoF04
    Participant

    Ah I get it (assigning modulators on panel initialisation. In the “calles when panel has finished loading” you put a method called PanelLoaded. In this method you call a function called “AssignModulators”where all the variables are set with their corresponding on panel modulators.

    This is clear and for a rebuild very handy. I’m thinking of making stuff more modular instead of coding it all 6 or 16 times (for the samples e.g.).

    Further more on the load in different steps. This is quiet difficult, I’m afraid. Because of the fact that I send a request for sample dump to the S700. Then the S700 starts sending sysex bytes. While sending it the S700 needs to receive an ACKS byte after sending the first 19 bytes. See what the implementation documentation is telling about the communication outline. (attachement)
    Fijne avond, insgelijks! :*)

    Attachments:
    You must be logged in to view attached files.
    #72577

    EnzoF04
    Participant

    Is a timer sync’ed to incoming bytes? Let’s say

    if MidiMessage:getSize() = 18 then
    timer = timer+1
    end

    I’m doing:

    	msgSize = MidiMessage:getSize()
    	msgAck = CtrlrMidiMessage({0xf0, 0x7e, 0x7f, 0xf7})
    
    	if msgSize == 19 then
    		panel:sendMidiMessageNow(msgAck)
    	end
    

    But i’m realising that the getSize is completed after the whole sysex msg is received? Am I right?

    • This reply was modified 4 months, 1 week ago by  EnzoF04.
Viewing 20 posts - 1 through 20 (of 36 total)

You must be logged in to reply to this topic.

Comments are closed.