Splitting Sysex for older synths

Home Forums General Programming Splitting Sysex for older synths

This topic contains 14 replies, has 5 voices, and was last updated by  m.tarenskeen 6 hours, 13 minutes ago.

Viewing 15 posts - 1 through 15 (of 15 total)
  • Author
    Posts
  • #84239
    vernonr
    vernonr
    Participant
    • Topics: 1
    • Replies: 5
    • Total: 6

    I have a wavestation SR – and would like to use CTRLR to create an editor/librarian for it. So far I have found most things possible – excellent. But there seems to be a snag sending the syses dump of a bank to the wavestation. The bank is around 64K. Ctrlr is easily able to send it – and the monitor confirms it can send it all in one message. The screen on the wave station shows receiving sysex but then sticks on a timeout.
    My suspicion is the transmission is too much, too fast – and overruns the receiving buffer.

    I can’t find between any way from a lua programme to split the message into multiple buffers with a time delay between each successive buffer. Not even sure I could do it from the ctrlr C++ code. It looks as though it might even be missing from JUCE.

    Have I missed something – or is this a current limitation?

    Wavestation was brilliant - inside the box - the challenge was always getting that visible.

    #84240

    human fly
    Participant
    • Topics: 123
    • Replies: 1036
    • Total: 1159
    • ★★★★

    Goodweather and Possemo will be able to tell you all about that.
    am not familiar with it, but it will involve a timer callback.
    maybe you will find what you need in the Pro2 panel. or perhaps
    the matrix1000 panel.

    #84241
    vernonr
    vernonr
    Participant
    • Topics: 1
    • Replies: 5
    • Total: 6

    Thanks

    I appreciate the stuff about timers and callbacks – and whilst that gives me the periodic action I want – it doesn’t actually send sequence of midi messages. It appears sendMidiMessageNow expects to send a single message that starts with a F0 and ends with a F7 – it seems to have a problem with blocks that don’t end with a F7 and the next send doesn’t happen..

    Wavestation was brilliant - inside the box - the challenge was always getting that visible.

    #84242
    Possemo
    Possemo
    Participant
    • Topics: 12
    • Replies: 418
    • Total: 430
    • ★★

    The synths I made panels for just have small sysex dumps so I never had to split messages. But maybe one day I will be faced with that problem too…

    I guess your dump is “one piece” beginning wiht F0 and ending wit F7

    IMHO the best way would be making a memoryblock out of your dump if not already done then selecting parts with getRange. I am not sure if it will work as a conform sysex dump needs the start and stop byte (F0/F7) but in general Ctrlr sends anything to MIDI-out when doing it by Lua scripts:

    DumpData = MemoryBlock()
    firstSplit = DumpData:getRange(0,256) — values explanation => (Starting Byte, Number of Bytes)
    secondSplit = DumpData:getRange(254,256) — values explanation => (Starting Byte, Number of Bytes)

    panel:sendMidiMessageNow(CtrlrMidiMessage(firstSplit))
    os.execute(sleep(20))
    panel:sendMidiMessageNow(CtrlrMidiMessage(secondSplit))

    Edit: ok. you already tried that… I was too late. How about the “Midi Global Delay”, have you tried it? It’s probably no solution either. This seems to be a real problem.

    • This reply was modified 1 week, 5 days ago by Possemo Possemo.
    #84244
    vernonr
    vernonr
    Participant
    • Topics: 1
    • Replies: 5
    • Total: 6

    Hi Possemo,

    Thanks. Tried that – the splitting works fine.. as does the loop. I’m pretty sure the first call to sendMidiMessage gets sent – the monitor confirms (and the synth says “Receiving Sysex” – but never sees the following packets – and finally fails with “Timeout”.
    My suspicion is that
    EITHER
    the wavestation can only cope with a few K in each lump, transferring each and assembling them elsewhere. A final confirmation of the checksum happens when the F7 is received..and controls whether the wavestation transfers it into the RAM bank and shows a success message following the transfer
    OR
    the wavestation wants the rate at which the data is sent to be much slower – and ctrlr outstrips the processing speed of the receiving circuitry.

    FWIW In the past I’ve had success with other software (Send SX BOME?) transmitting these banks to the wavestation. But I have no idea what went on inside the software.

    Wavestation was brilliant - inside the box - the challenge was always getting that visible.

    #84245
    Possemo
    Possemo
    Participant
    • Topics: 12
    • Replies: 418
    • Total: 430
    • ★★

    Ok, so these sysex dumps are correct. It is strange as normally you cannot vary the speed MIDI data is transmitted. It is defined by the MIDI specifications (MIDI has a single baud rate: 31250 baud – taken form there: http://www.personal.kent.edu/~sbirch/Music_Production/MP-II/MIDI/midi_physical_layer.htm).

    Have you checked the data being sent with a MIDI monitor software like MIDIoX? Maybe the delay set with os.execute(sleep(20)) has to be right. It could be that the synth won’t wait forever for the data to continue.

    #84246
    Possemo
    Possemo
    Participant
    • Topics: 12
    • Replies: 418
    • Total: 430
    • ★★

    It tested it and indeed it does not work. sendMidiMessageNow wants the first byte to be f0 otherwise it won’t send it.
    I will have a look at Sounddiver’s Wavestation panel (I guess it does have one). Sounddiver has showed me a lot of tricks in the past.

    • This reply was modified 1 week, 5 days ago by Possemo Possemo.
    #84249
    vernonr
    vernonr
    Participant
    • Topics: 1
    • Replies: 5
    • Total: 6

    Hi,

    Great test, your finding fits the symptoms I’ve observed. Initial packet starting with F0 sent – but no option to get anything afterwards sent.

    Let me know if sound diver has any clues. I’ll have a look see at snoize – which has source code available.

    On another tack I have been trying to find out whether I can find a way from LUA to execute sendMidiBuffer. This call appears to accept a set of messages. Don’t whether this tactic would be a viable workaround, or simply run into the same problem (if it actually is in JUCE).

    Sorry about red herring of transmission speed – silly me..

    Vernon

    Wavestation was brilliant - inside the box - the challenge was always getting that visible.

    #84253
    Possemo
    Possemo
    Participant
    • Topics: 12
    • Replies: 418
    • Total: 430
    • ★★

    I have now looked into a wavestation dump (64k) with the help of a hex editor. This dump consists in fact of 3 sysex messages. So it is clear what you have to do. Seperate the messages and set a delay in-between. When sysex messages are sent without delay many synths/systems will make problems.

    #84254

    dasfaker
    Keymaster
    • Topics: 79
    • Replies: 772
    • Total: 851
    • ★★★

    The sysex file you are trying to send, has only one F0 and one F7 (one single block of data) or has several blocks? According to these links, it’s made of several blocks.
    http://www.korgforums.com/forum/phpBB2/viewtopic.php?t=7550
    https://www.soundonsound.com/techniques/korg-wavestation-tips-techniques

    If that’s the case, you need to send them individually with a timer to allow a delay between them.

    #84255

    dasfaker
    Keymaster
    • Topics: 79
    • Replies: 772
    • Total: 851
    • ★★★

    You beat me Poss!!

    #84256
    vernonr
    vernonr
    Participant
    • Topics: 1
    • Replies: 5
    • Total: 6

    Hi Both,

    I’ll try that and let you know.

    Thanks

    Wavestation was brilliant - inside the box - the challenge was always getting that visible.

    #84262
    Possemo
    Possemo
    Participant
    • Topics: 12
    • Replies: 418
    • Total: 430
    • ★★

    Ah. for once I was faster than dasfaker.. 🙂

    I would prefer os.execute(sleep(value in milliseconds)) instead of a timer. sleep works fine – just be aware that the MIDI Monitor of Ctrlr will display wrong values. It looks like sleep has no effect but in fact when you look at it with MIDIOX you can see that it does work flawlessly.

    #84366
    vernonr
    vernonr
    Participant
    • Topics: 1
    • Replies: 5
    • Total: 6

    Hi,

    Have managed to do some further tests. I succeeded in sending large sysex messages to the Wavestation SR using SX Send (Bome Software) having resurrected an old windows 2003 server machine.

    Each Patch bank sysex message is 29K. The Performance Banks are 17K. and the wave sequences 17K (so 9 in total for the RAM banks of the SR). But the important lesson is how SX Send offers to send the message. SX Send offers a slider to adjust the speed of the message. I got it working sending by setting the slider to 1300 bytes/second. It counted the lines of 64 byte sysex (as shown on the screen) sent. That (as mentioned above) isn’t close to the 31,500 baud rate specified by the Midi standard. So SX Send would appear to be splitting the message into much smaller buffers and send these individually with a configurable pause between each message.

    This fits with the way Snoize SYSEX librarian (a Mac program) appears to try and do it – and the source code refers to the MIDIOX practice of splitting the sysex message into 256 Byte buffers. This may have been driven by windows 16 bit nature..

    I have yet to test sending these sysex messages using Snoize Sysex librarian which would be good – given that the source code can be inspected/changed – unlike SX Send which is a binary compiled app.

    So earlier thoughts about splitting messages appear to be right. Ctrlr can’t yet split messages into sufficiently small lumps because it won’t send buffers that don’t start with F0. I think it’s a change to LUA/Ctrlr infrastructure that we need.

    Cheers

    Wavestation was brilliant - inside the box - the challenge was always getting that visible.

    #84367

    m.tarenskeen
    Participant
    • Topics: 25
    • Replies: 95
    • Total: 120
    • ★★

    Ah. for once I was faster than dasfaker.. 🙂

    I would prefer os.execute(sleep(value in milliseconds)) instead of a timer. sleep works fine – just be aware that the MIDI Monitor of Ctrlr will display wrong values. It looks like sleep has no effect but in fact when you look at it with MIDIOX you can see that it does work flawlessly.

    Thanks, I didn’t know that. I can use that in one of my homemade panels. For me the timer construction is difficult to grasp. Using os.execute(sleep()) is easier to understand. But I can imagine situations where sleep can not be used. For example when used in situations where a track is playing in a DAW simultaneously. Timing will be ruined then, or maybe the system will crash?

Viewing 15 posts - 1 through 15 (of 15 total)

You must be logged in to reply to this topic.

There is currently 1 user and 11 guests online
miffebarbez
Forum Statistics
Threads: 2,131, Posts: 14,985, Members: 10,033
Most users ever online was 5 on March 28, 2018 6:11 pm
Do NOT follow this link or you will be banned from the site!