Not sure what you mean with "does not work", but here are a few things:
Your regular expression isn't bad, but it's not perfect. I suggest you use PHP's built in function for that.
*Not* allowing characters in passwords is a bad idea. Don't discourage the users to use safer passwords.
All the hours you've spent on thinking about the security and complexity of your script are worthless unless you escape your user input before throwing it into a SQL query. For your own good, read and apply this:
PHP: mysql_real_escape_string - Manual
I'd suggest moving away from the old mysql_* extension at all. It's very old, insecure, and encourages bad coding practices. Take a look at
PDO and
MySQLi (improved MySQL extension). They're a little more complicated, but you won't regret it.
One last thing, don't use the ENCODE() function to encode user passwords. Passwords should be hashed, and not encrypted. At no point you need to know what the original password was. Use PHP's
md5() function instead.