handing bitmapped MIDI parameters

Home Forums General Programming handing bitmapped MIDI parameters

Viewing 19 posts - 1 through 19 (of 19 total)
  • Author
    Posts
  • #118581
    rheslip
    Participant
      • Topics: 1
      • Replies: 8
      • Total: 9

      The synth I’m programming (XFM2) has about 10 parameters that are packed into bitmaps. I have a UI button for each bit in the bitmap. I’m using global variables k1, k2 etc to store the bitmaps and expressions like:
      setGlobal (1, setBit (global.k1, 1, modulatorValue))

      to merge the bitmapped parameters into a byte that can be sent via sysex to the synth. This works. The problem is syncing the UI to the synth.

      When I load a patch from the synth I’m syncing the UI buttons with a Lua script that finds the buttons by name and sets them from the bits in the MIDI SYSEX. For this to work correctly I also have to set the global to the value from the SYSEX. I have not been able to find a method to set a global variable other than

      void CtrlrPanel::setGlobalVariable(const int index, const int value)

      Obviously I can’t use this with a variable because its expecting constants.

      I wrote some LUA code to merge the bitmaps into a Lua array. The bitmap merging part worked but it caused a race condition when I tried to update the UI – updating the UI buttons made it send MIDI messages to update the bitmapped parameters again and everything gets scrambled.

      I’m stumped. Any ideas would be appreciated. Thanks.

      #118583
      dnaldoog
      Participant
        • Topics: 4
        • Replies: 480
        • Total: 484
        • ★★

        What function are you using to update the UI?

        #118590
        Possemo
        Participant
          • Topics: 14
          • Replies: 638
          • Total: 652
          • ★★★

          I did use global variables in one of my panels. You can set them like this:

          panel:setProperty("panelGlobalVariables",m1..":"..m2..":"..m3..":"..m4..":"..m5..":"..m6..":"..m7..":"..m8..":"..m9..":"..m10..":"..m11..":"..m12..":"..m13..":"..m14..":"..m15..":"..m16..":"..m17..":"..m18..":"..m19..":"..m20..":"..m21..":"..m22..":"..m23..":"..m24..":"..m25..":"..m26..":"..m27..":"..m28..":"..m29..":"..m30, false)

          m1 to m30 are variables, m1 is the first one, e.g. k1… etc.

          #118591
          rheslip
          Participant
            • Topics: 1
            • Replies: 8
            • Total: 9

            I fixed the race condition by using a timer to disable MIDI output while the UI is updating.

            This is the loop I’m using to update the UI buttons for each bitmapped parameter. Each button has a name of the form “1bit1” which means it is sysex parameter 1 bit 1.

            for j=0,7 do — set the modulators from the bitmap
            local bitval=bit.band(midiParam, 1)
            name = string.format(“%dbit%d”, paramnumber,j) — find the modulator for this bit
            mod = panel:getModulatorByName(name)
            if mod ~= nil then
            mod:setModulatorValue(bitval, false, false, false)
            end
            midiParam=midiParam/2 — bit shift right
            end

            I spent a lot of time trying different things yesterday and concluded that LUA is not behaving correctly. eg bit.band (0xff,0x1) results in 0xff, if then else clauses not working, code that used to work no longer works etc. Just tried installing CTRLR 6.04 and it crashes when I run the problematic LUA code. The code was developed with 5.4.52 win64.

            Maybe my panel is corrupted? I suspect I have duplicated modulators from cutting and pasting. Any ideas on how to clean up the XML?

            Thanks

            #118592
            rheslip
            Participant
              • Topics: 1
              • Replies: 8
              • Total: 9

              Thanks for the tip on setting globals. That should work. In the meantime I went back to the code that uses a LUA array to store the bitmaps because it ~ALMOST~ works. If I use globals I have to go back and change ~400 UI properties which I would like to avoid.

              #118593
              Possemo
              Participant
                • Topics: 14
                • Replies: 638
                • Total: 652
                • ★★★

                I would use the recommended stable version v5.3.201 …could save you some headaches.

                btw. the XFM2 looks like one great project. Building yourself an enhanced DX7 which sounds great (listened to youtube) for just under 100$. But with over 450 parameters it won’t be the easiest editor to make… I wish you all the luck.

                #118594
                dnaldoog
                Participant
                  • Topics: 4
                  • Replies: 480
                  • Total: 484
                  • ★★

                  Hi rheslip,

                  Using a timer to disable MIDI is one way. I am guessing you are using

                  panel:setPropertyInt("panelMidiPauseOut",1)

                  then

                  panel:setPropertyInt("panelMidiPauseOut",0)

                  You could also prevent the modulator triggering by substituting

                  mod:setValue(bitval, false)

                  for

                  mod:setModulatorValue(bitval, false, false, false)

                  RE: forums/topic/setmodulatorvalue-and-friends/
                  In my opinion setModulatorValue() is ‘wonky’ for want of a better word.

                  On 5.3.201

                  x=bit.band(0xff,0x01)
                  console(String(tostring(x)))

                  returns ‘1’ for me?

                  To delete or rename modulators go to Panel->Modulator list, but be careful, because if you delete too many at once, Ctrlr may crash.

                  Not sure what you are doing exactly, but using a lua lookup table as you said you reverted to, would probably be the best way to manage so many modulators. Would be interested to see the panel so far.

                  • This reply was modified 3 years, 9 months ago by dnaldoog. Reason: fixed errors
                  #118596
                  rheslip
                  Participant
                    • Topics: 1
                    • Replies: 8
                    • Total: 9

                    The XFM2 sounds amazing. Its quite an easy build and its cheap. I found a few odd things but 99% of it works as advertised. A DX7 on steroids!

                    I based my XFM2 panel on the DX7 panel by Sjoerd Bijleveld, Torsten Tittmann and Pascal Collberg. Used most of the same UI elements and some of the Lua code.

                    I brute forced the MIDI message suppression with a flag and a timer. I didn’t realize you could do it with built in methods. CTRLR is cool but it seems you have to read a lot of source code to find these things out.

                    I did a lot of cutting and pasting on this panel because there are six identical groups of operator controls. I noticed its easy to “lose” UI elements if you are not careful dragging them into position. I think this may be my problem – I’m guessing there are phantom UI buttons that are overwriting the ones I think I’m updating. I noticed the button text which should read “ON” or “OFF” is greyed out on one button after I refresh the UI. If I manually toggle it goes back to normal. I found and removed a couple of duplicated UI elements by searching the XML but when I update the UI those buttons are still not updating properly.

                    I really wish I had started with v5.3.201. Now I find I can’t go back – CTRLR goes a little wonky when I load this panel with v5.3.201. All the evidence suggests the later versions I’ve tried are unstable. I REALLY do not want to start over! If somebody knows how to downgrade from later releases I would like to do that.

                    Thanks for the suggestions!

                    #118602
                    Possemo
                    Participant
                      • Topics: 14
                      • Replies: 638
                      • Total: 652
                      • ★★★

                      How does unstabiltiy manifests with 5.3.201? In general you should be able to go back to 5.3.201.

                      Ctrlr should be able to handle several modulators having the same name, but there may arise problems if they are updated by Lua code. I dont know as I clean them up in my panels (a good idea anyway). The modulator-list is quite helpful for this, Menu: Panel->Modulator list.

                      Editing the xml-file is tricky, I would avoid it unless there is no other way.

                      When you are updatin the ui, could it be that the button gets a value beyond its range? A button expects value 0 or 1. If you need the button to send other values you should set them like this:
                      OFF=32
                      ON=64
                      But to update the button you still have to use 0 and 1.

                      Buttons are another thing in Ctrlr that is wonky. You can make Ctrlr work flawlessly but you have to work around bugs…

                      #118589
                      Possemo
                      Participant
                        • Topics: 14
                        • Replies: 638
                        • Total: 652
                        • ★★★

                        I used global vars in one of my panels. You can set them like this:

                        panel:setProperty("panelGlobalVariables",m1..":"..m2..":"..m3..":"..m4..":"..m5..":"..m6..":"..m7..":"..m8..":"..m9..":"..m10..":"..m11..":"..m12..":"..m13..":"..m14..":"..m15..":"..m16..":"..m17..":"..m18..":"..m19..":"..m20..":"..m21..":"..m22..":"..m23..":"..m24..":"..m25..":"..m26..":"..m27..":"..m28..":"..m29..":"..m30, false)

                        Here I did set 30 vars – named m1 to m30.

                        The XFM2 looks very interesting. I first wanted to ask you what you are meaning by “bitmapped parameters” but then I read about this fully featured 64-voice 6op FM-Synth. In the MIDI implementation chart I see as much as 454 parameters! Instead of fixed algorithms you can build your algo yourself with this bitmapped parameter… great!

                        Maybe other people will be interested in this astonishing project: build yourself an uber-dx7 for less than $100.
                        https://www.futur3soundz.com/xfm2

                        And it sounds damn good:
                        https://www.youtube.com/watch?v=fJyIRA1ev-A

                        Edit: this is an old post – obsolete by now. I did’t realize that you must not include links in a post.

                        • This reply was modified 3 years, 9 months ago by dasfaker.
                        • This reply was modified 3 years, 9 months ago by Possemo.
                        #118597
                        rheslip
                        Participant
                          • Topics: 1
                          • Replies: 8
                          • Total: 9

                          This is what the XFM2 panel looks like at this point. Not pretty but it works – except for the OP buttons which update incorrectly when I sync the panel to the synth.

                          XFM2 Operator control page

                          XFM2 global controls and effects

                          #118610
                          Possemo
                          Participant
                            • Topics: 14
                            • Replies: 638
                            • Total: 652
                            • ★★★

                            Pictures do not show. Can you post a .bpanelz file? Menu File -> Export -> Export compressed binary + ressources.

                            I once had an idea making a panel for my TX81Z which would move the operators when changing alogrithm – placing them in the panel to show the selected algorithm. It would be some work but feasible. Well on the XFM2 which allows connecting the 6 ops freely that would be quite challenging…

                            #118612
                            rheslip
                            Participant
                              • Topics: 1
                              • Replies: 8
                              • Total: 9

                              I’ll try dropbox links:

                              XFM2 Operator page (image)
                              XFM2 Global controls and effects page (image)
                              panel file

                              I don’t want to post the panel on the CTRLR panel page yet because its not feature complete and it has this annoying bug with the OP buttons. The OP buttons work correctly for editing the XFM2 – there are 64 possible algorithms times 6 ops. What does not work correctly is setting the buttons when I pull a patch from the XFM2. It works correctly if there is only 1 OP button set, but shifts them if more than one is set. Its very strange.

                              I am setting the button values to either 0 or 1. I’m not using bit.band() anymore because the results were incorrect – more evidence that something is really messed up. I have a couple of lines of LUA that extracts the least significant bit of the bitmap using mod arithmetic and I shift right using division.

                              If I install 5.3.201 I can load other panels OK but when I load the XFM2 panel the whole CTRLR UI changes – text gets larger, the panel tabs have a large X beside them. It doesn’t crash CTRLR but the XFM2 panel doesn’t work either.

                              I would start over with 5.3.201 but creating those panels took a LOT of time. I have probably made a mess of this thing anyway.

                              #118616
                              dnaldoog
                              Participant
                                • Topics: 4
                                • Replies: 480
                                • Total: 484
                                • ★★

                                The coding (and panel) looks very nice, but I am surprised lua bit.band etc functions aren’t working. You can also do bit masking with JUCE class BigInteger() too.

                                
                                shiftBits()
                                void BigInteger::shiftBits	(	int 	howManyBitsLeft,
                                int 	startBit 
                                )		
                                Shifts a section of bits left or right.
                                
                                Parameters
                                howManyBitsLeft	how far to move the bits (+ve numbers shift it left, -ve numbers shift it right).
                                startBit	the first bit to affect - if this is > 0, only bits above that index will be affected.
                                
                                

                                I haven’t tried this function (shiftBits) but looks interesting. I guess good old division or multiplication works anyway.

                                Also I noticed you are using variables starting with digits. One uiSlider has the name 7 among many others starting with digits. I don’t think that’s legal in lua, so it could be something that might trip you up. Personally I also don’t use the – symbol in modulator names as it is not legal in lua either.

                                Also

                                
                                               mod:setModulatorValue(midiParam, false, false, false)
                                

                                …in my experiences gives unpredictable results. Unfortunately I can’t give a concrete example, but I do remember once being very confused as to why something wasn’t working and it turned out changing to mod:setValue(midiParam,false) or mod:getComponent():setValue(midiParam,false) or whatever it was worked.

                                I also seem to remember having problems with the inbuilt Global Variable functionality setGlobal (1, setBit (global.k1, 6, modulatorValue)) so I ditched that route in my panels, but if Possemo says it works then it must work.

                                That’s clever using %2 to find the bit 1 LS bit (didn’t know that).

                                See attached panel showing bit.band working alongside mod%2 and BigInteger()

                                • This reply was modified 3 years, 9 months ago by dnaldoog. Reason: typos
                                Attachments:
                                You must be logged in to view attached files.
                                #118619
                                rheslip
                                Participant
                                  • Topics: 1
                                  • Replies: 8
                                  • Total: 9

                                  Thanks for the suggestions!

                                  I have tried changing the button names so they start with alpha characters op1bit0 etc but it still didn’t work. I will try the other forms of setModulatorValue() and I’ll clean up the names too.

                                  FYI: the creator of the XFM2 Rene Ceballos has just released an alpha of a new FPGA synth that runs on the same $100 hardware. Its a subtractive synth with 32 voice polyphony, 4 osc per voice, 3 env gens, 2 lfos, stereo audio and the same wonderful FX chain as the XFM2. I have been playing with it and it sounds pretty good. He reused a lot of the XFM2 blocks so I should be able to create a CTRLR panel for this one fairly easily.

                                  XVA1 Virtual Analog FPGA synth

                                  You can bet I will be using the stable release of CTRLR to develop that editor!
                                  Rich

                                  #118622
                                  dnaldoog
                                  Participant
                                    • Topics: 4
                                    • Replies: 480
                                    • Total: 484
                                    • ★★

                                    Hi rheslip,

                                    Also this is another recommended optimising structure and the main reason why one would need legal lua variable names.

                                    In a panelCreated or before PanelCreated function initialise all your Modulators:

                                    op1bit0 =panel:getModulatorByName("op1bit0 ") etc

                                    Hereafter you can refer to that modulator by the variable name, not panel:getModulatorByName(“op1bit0 “) because much slower apparently. For each call to that, Ctrlr loops through all Modulators looking for that object.

                                    Here is a development function to help you generate the list:

                                    Copy and paste the console output into the init function.

                                    
                                    function cacheModNames()
                                      n = panel:getNumModulators()
                                      --Assign each modulator to a variable with same name
                                      --so we can just call myModulatorsName:getValue() etc
                                      console("\n\n")
                                      local _t={}
                                      for i=0,n-1 do
                                        mod = panel:getModulatorByIndex(i)
                                        local Name= L(mod:getName())
                                        table.insert(_t,Name)
                                      end
                                      table.sort(_t)
                                      for i,v in ipairs(_t) do
                                        console(String(string.format("%s=panel:getModulatorByName(\"%s\") --[%d]",v,v,i)))
                                      end
                                      console("\n\n")
                                    end --function
                                    ---------------------------------------------------------
                                    

                                    Incidentally, the panel looks fine to me on Win 10 5.3.201

                                    #118626
                                    rheslip
                                    Participant
                                      • Topics: 1
                                      • Replies: 8
                                      • Total: 9

                                      Excellent suggestion! The current implementation creates the name on the fly from the loop variables which is slower but less code. If I was to unroll the loops that would work very well which I may yet do. There are about a dozen bitmapped parameters so unrolling the nested loops is not too bad and it might miraculously fix my problem.

                                      So the XFM2 panel works for you under Win 10 5.3.201 – very interesting. I developed it on my old Win 7 desktop which has been showing signs of instability lately. I need to upgrade the machine anyway, this is more motivation. I could be chasing OS related problems.

                                      Thanks a lot for your help dnaldoog!

                                      Rich

                                      #118656
                                      rheslip
                                      Participant
                                        • Topics: 1
                                        • Replies: 8
                                        • Total: 9

                                        I finally got bitmaps working using global variables. Each of the six XFM2 operators has 7 buttons that are mapped to the bits in global variables 1 to 6. OP1 bits are stored in k1, OP2 bits in k2 etc. Each button has an expression in the property panel which toggles its bit in the global e.g. to set bit 1 in k1:

                                        setGlobal (1, setBit (global.k1, 1, modulatorValue))

                                        Each button has a name of the form “p1bit1” which means parameter 1 bit 1 (sysex parameter 1 in the XFM2 happens to be operator 1). When I receive a sysex patch dump from the synth I sync the state of the OP buttons using this ugly code:

                                        if paramnumber == 1 then
                                        op1=midiParam — save OP bits to update globals
                                        mod = panel:getModulatorByName(“p1bit0”)
                                        if mod ~= nil then
                                        if bit.band(midiParam,1) == 1 then
                                        mod:setModulatorValue(1, false, false, false)
                                        else mod:setModulatorValue(0, false, false, false)
                                        end
                                        end

                                        mod = panel:getModulatorByName(“p1bit1”)
                                        if mod ~= nil then
                                        if bit.band(midiParam,2) == 2 then
                                        mod:setModulatorValue(1, false, false, false)
                                        else mod:setModulatorValue(0, false, false, false)
                                        end
                                        end

                                        etc etc for each bit of each bitmapped parameter. Not elegant code but the elegant approach didn’t work. After all that I sync the globals to the parameter value which was saved above:

                                        panel:setProperty(“panelGlobalVariables”,Unit..”:”..op1..”:”..op2..”:”..op3..”:”..op4..”:”..op5..”:”..op6, false)

                                        “Unit” is a variable I keep in sync with global k0. Its used in sysex messages to address the XFM2 unit number.

                                        Other things I had to do:
                                        – go back to stable release 5.3.201 – I had all kinds of strange LUA behavior with later versions
                                        – had to recreate the entire UI using “copy with children” to cut and paste subpanels from an older version of the panel to the newly created one. I did this because the panels I had created with later versions of CTRLR would not work correctly under 5.3.201
                                        – CTRLR bug in copy/paste resulted in a whole pile of duplicated modulators which I had to find and delete using the modulator list. They generally show up as modulators with the same name in a phantom componentGroupName that is invisible. Quite tedious.
                                        – I’m still using numbers as modulator names which is “illegal” in Lua but it works. It makes finding the modulator by its sysex parameter number quite simple. That helps a lot because the XFM2 has over 450 parameters…

                                        It took about ten times longer than it should have but now I have a panel for the XFM2 that is working quite well. Still have to implement file load/save and a few other things but when I’m happy with it I’ll post it on the panel page.

                                        Thanks a lot for the suggestions and many thanks to the authors of the DX7 panel for sharing their work. The XFM2 panel is based on their UI and code.

                                        Rich

                                        #118659
                                        dnaldoog
                                        Participant
                                          • Topics: 4
                                          • Replies: 480
                                          • Total: 484
                                          • ★★

                                          I look forward to seeing it!

                                        Viewing 19 posts - 1 through 19 (of 19 total)
                                        • The forum ‘Programming’ is closed to new topics and replies.
                                        There is currently 0 users and 51 guests online
                                        No users are currently active
                                        Forum Statistics
                                        Threads: 2,495, Posts: 17,374, Members: 77,605
                                        Most users ever online was 12 on January 22, 2019 3:47 pm
                                        Ctrlr