PHP/MySQL Experts : How to Generate a Unique Sequence of Numbers?

by 26 replies
36
I originally thought this would be quite simple but it now appears far from trivial. Perhaps I am missing something.

I want a way to create a sequence of 32-bit tokens that does not repeat and must be guaranteed unique across different sessions.

Preferably pseudo-random but sequential would be OK too.

Although MySQL does not support sequences one possible way is to create a table with an auto_increment column and for a new token insert NULL and then read back last_insert_id, but I would prefer to avoid an extra database write per session.

Any other ideas?

The tokens must be 32-bits long and the method work consistently on 32/64-bit servers without requiring any extensions or privileges beyond a typical default hosting account.

I'm beginning to think the above method may be the only way and would be interested in the thoughts of some more experienced PHP/MySQL developers.

Cheers,

Phil
#programming #experts #generate #numbers #php or mysql #sequence #unique
  • Ok you have to keep track of each session increasing your number and based on that continue with the next binary number, I mean session 1, [1 .. 32] session 2 [33 ..64] session 3 [65 .. 96] and so on, so for each number you have to return an array of numbers as described above and then convert them from decimals to binary and from binary to chars, that is it.
    Now let's make the formula, if N is any number (1,2,3,...)
    base=(N-1)*32;
    A[1]=base+1;
    A[2]=base+2;
    ...
    A[32]=base+32;
    That will make it
    Good luck
    • [1] reply
    • Thanks for the reply, although I admit I don't understand the method you are suggesting.

      Maybe I didn't explain well enough. There is nothing special about the tokens, they are just numbers.

      If each session could get the new value from a sequence 1,2,3... that would be fine, I could hash them to make them look random.

      The problem is really twofold:

      1) Finding a unique sequence generator that doesn't repeat, but if it's sequential and not pseudo-random then that's not an issue.

      2) Atomically selecting the next value from the sequence across multiple simultaneous sessions.

      The MySQL auto_increment method is the only one I could come up with. The problem with that is it's impossible to get the next number without writing to the database first. Guess I will just have to take the hit.

      Cheers,

      Phil
  • Why you don't take the seconds of the Greenwich time as a base plus milliseconds?
    I mean the seconds from 1970 up to the moment.
    • [ 1 ] Thanks
    • [1] reply
    • It would be great to use time and I did consider this, but millisecond precision would steal 10 bits and leave only 22 for the seconds which would wrap in less than two months, and that's a bit soon.

      The chances of a collision are probably quite slim but I still wouldn't rest easy.

      If the token were larger, say 64-bits, MySQL has a UUID_SHORT function which is guaranteed unique.

      Hopefully one day sequences will be built into MySQL. Until then I still think the auto_increment column is the only sensible way forward. Either that or increase the token size.

      I miss PostgreSQL and Python. Looking forward to seeing them again.

      Cheers,

      Phil
      • [1] reply
  • $unique_token = md5(time().microtime());

    Gleb
    • [ 1 ] Thanks
    • [ 1 ] Thanks
    • [1] reply
    • Thanks Kirk,

      That's an interesting idea although it's a fair bit of overhead just to get a number and I am not sure about its atomicity.

      Also not sure if function creation is permitted by default on a typical shared host.

      I think it's aimed more at a compatibility layer for PostgreSQL sequences?

      Cheers,
      Phil
  • I was thinking about this post, and queried this: crc32. I wasn't even aware of this hash type, but apparently it's been available in PHP for quite some time now. According to this page, the chance of a random collision is 1 / 2^32
    • [1] reply
    • Hi,

      I think there may be some confusion about CRC32 on the page you referenced.

      As it's not a 'perfect hash' a CRC32 will generate the same output even for many different inputs, and the chances of collision are much, much higher than the figure mentioned.

      I think generally a common point of confusion is the difference between 'random' and 'unique'.

      A good random number generator will repeat some numbers before using all the numbers in its range. That's what random is - each new selection is independent of all the previous selections, so having repeating numbers is normal and desirable for an RNG.

      A 'unique' number generator is a little different. The simplest of course is to just start counting 1,2,3... but if we want the numbers to be unpredictable then what is really needed is a shuffle where each number will only be used once. See the discussion I had with Anthony for some ways to achieve this.

      A unique sequence may look random, and the choice of number may be (pseudo) random, but the sequence itself is certainly not random.

      Even with 2^32 unique inputs, a CRC32 would repeat many, many times and the collision rate can be no better than random numbers which I still think would generally average around 1 in 1000 for 32-bits.

      Cheers,
      Phil
  • Here you go man. If you just want numbers modify the function and remove the letters.

    <?
    //GET A RANDOM STRING
    echo createRandomString('25');


    //*****************************
    //FUNCTION: CREATE RANDOM STRING
    //*****************************
    function createRandomString($stringLength) {
    $chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVW XYZ0123456789";
    srand((double)microtime()*1000000);
    $i = 0;
    $pass = '' ;
    while ($i <= $stringLength) {
    $num = rand() % 33;
    $tmp = substr($chars, $num, 1);
    $randomNum = $randomNum . $tmp;
    $i++;
    }
    return $randomNum;
    }

    ?>

    Let me know if you need any help with it.
    Best of luck!
    • [ 1 ] Thanks
    • [1] reply
    • //*****************************
      //FUNCTION: CREATE RANDOM STRING
      //*****************************
      function createRandomString($stringLength) {
      $chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRS TUVW XYZ0123456789";
      srand((double)microtime()*1000000);
      $i = 0;
      $pass = '' ;
      while ($i <= $stringLength) {
      $num = rand() % 33;
      $tmp = substr($chars, $num, 1);
      $randomNum = $randomNum . $tmp;
      $i++;
      }
      return $randomNum;
      }




      Hi, and thanks for your reply.

      Although using a PRNG is not suitable for my application I do have a few thoughts on the code you supplied and some suggestions for improvements if you are interested.

      1) The number of selectable characters in the code you posted is 62 but the code is only selecting from the first 33.

      2) PHP's rand() function is really a terrible PRNG. On some platforms it's default range is only 15-bits. mt_rand() using the Mersenne Twister algorithm is better, but still only 31 bits on some platforms, I believe.

      3) Seeding is done automatically since PHP 4.2. Reseeding every time is not necessary and probably won't make things any more random.

      4) If you rearranged the selectable string a bit you could have a generic function for generating strings of random numbers for any base from 1 to 62.

      Here's a function I use for generating pseudo-random keys, passwords and numbers of different bases.

      function _random_key($length,$base) {
      $chars = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklm nopqrstuvwxyz';
      $key = '';
      for ( $i=0; $i<$length; $i++ ) {
      $key .= $chars[ (mt_rand( 0, ($base-1) ))];
      }
      return $key;
      }

      (sorry about the formatting - the forum barfed on php or code tags, so plain text it is. That space after the 'm' in the chars string isn't really there either.)

      So for a random 32-bit number as hex...

      _random_key(8,16)

      or whatever length and base you want, up to base 62.

      Cheers,

      Phil
      • [ 1 ] Thanks
      • [1] reply
  • That code looks very familiar bucksuper... lol

    It's almost exactly the same as the code I had in SMP to generate random passwords. It looks like an advanced workup of what I had.. but really nice use of a random character generator.
    • [1] reply
    • Johnny,
      What's SMP?
      I don't think I wrote that from scratch but I definetely modded it. Not sure but I've been using it for years.
  • Buck, sorry haven't been in this area of the forum for a few days and missed your question. SMP is Simple Member Pro. It's a membership script I wrote 3 years ago.

    It does use some code from another script that I had PLR to, namely the admin and member logins and html formatting. The code you posted was almost line for line the same code that is in the common class that was in the code I modeled my script after. A lot of people do have rights to that particular block of code so it wouldn't surprise me if you did see it in another script some time in the past.
    • [1] reply
    • Guys I am looking for someone to Modify the Butterfly script to not include the the JV partners with my normal Autoresponder List.

      If one of you would be interested or knows someone that would be please let me know I appreciate the help.

      Chris

Next Topics on Trending Feed