Nov-10-2016, 10:21 PM
There was a thread recently about checking passwords against previously known values. Passwords (and verification answers, email addresses sometimes, ssn, phone numbers etc) are information which you use to verify that someone is who they say they are, but which you never actually need their answer for. You need to verify that what they type matches what they typed previously, but you don't ever need to see what that is. So you NEVER store the actual value anywhere.
Think about all the information you have across all the varied websites you use, and how bad it would be if that information was freely given to other people. Think about the worst case scenario: your database has been cracked wide open, and anyone can see any of the information stored there. The goal here, is to store things nobody would want to see anyway, so even in the face of a worst case scenario, nothing bad actually happens.
In order to accomplish this, we use one-way hashing algorithms. In software and various types of documents, you'll see something called an "md5 hash", so after you download the file you can hash it yourself and compare the hashes to make sure what you actually downloaded is what they made available (to protect yourself from man-in-the-middle attacks, or, more likely, just to make sure the downloaded files weren't corrupted).
Let's take a side trip, and pretend we're a l33t kracker with mad skillz, and we want to pwn some systmz. Understanding what an attacker has to do will help to understand why we store passwords the way we do. So, we've gotten into a database, and in the User table, we see there's a column called "Password". Bingo. But then you see the value of the column looks like this:
md5 and sha512 aren't actually good enough for passwords for a variety of reasons. They're too fast. Well, and md5 is broken (two different passwords can yield the same hash). Time is the enemy. If you give an attacker enough time, they can eventually unwind whatever insane algorithm you're using to encrypt your password, and then apply the same process to a rainbow table, then compare it with your database to get the passwords. So the longer something takes, the better it is. sha512 and md5 are designed to be fast.
Luckily, this is 2016 (almost 2017!). We have more than just some random loon proclaiming things are bad and other things are good. We have many loons. Science has come to the rescue, and delivered upon you a simple solution: use bcrypt. It's been tested hard for years, it's designed for passwords, it's slow on-purpose, and it's available for almost all languages. In python, it's as simple as "pip install bcrypt".
Think about all the information you have across all the varied websites you use, and how bad it would be if that information was freely given to other people. Think about the worst case scenario: your database has been cracked wide open, and anyone can see any of the information stored there. The goal here, is to store things nobody would want to see anyway, so even in the face of a worst case scenario, nothing bad actually happens.
In order to accomplish this, we use one-way hashing algorithms. In software and various types of documents, you'll see something called an "md5 hash", so after you download the file you can hash it yourself and compare the hashes to make sure what you actually downloaded is what they made available (to protect yourself from man-in-the-middle attacks, or, more likely, just to make sure the downloaded files weren't corrupted).
Let's take a side trip, and pretend we're a l33t kracker with mad skillz, and we want to pwn some systmz. Understanding what an attacker has to do will help to understand why we store passwords the way we do. So, we've gotten into a database, and in the User table, we see there's a column called "Password". Bingo. But then you see the value of the column looks like this:
> select Password from User; 0c18c2a2cf4c2bca864f948409c7295ce784a97d4c9e27bb69cdfcdf41dfa8a7051480b8ac6c8ea87d89c611af321ab2bc2e95aded966099feab6c3e19bde182:8090a9c58052ccfae4e0720341401ccde410af6213b1d197abd3ddffd3a47a9df8f8fe10a7227c1d5c4f8560a3c26efd69f84356eeb0ae6b6c1426ccb418dfd8 2a2586aeb1d2d6a035a9b49d84268970e420bcdc7ce28faaeb555fe55bf9f9df3f422e3a8559d09a6a1329eb1dbd86a89cb61aa259d8e301f41de23eab0fbfc6:c0b879f6c97e8a9899492915bf3e32bea6ce983ac97e8c4870fc74aae70471fb20d4e989e2cb1a4e5b35382eab6da7be08a5a17115005f037ae7288c877fc8eb a0a5d0c8391dfd784b91f261d1d03c1a5a9fdf576e98dbd0669526506d620a7e7b84a44c04890cf8492e05a2acfd61d6fb121d4c30be775d0aca58f18443f624:e0fa51466e340ee777bf2fda8640377cf11ecf58ed36935b6d05cc22860735b4414152cb468aae766038c8deb01d79ab950a2205fa60e1c05432376a3fec0557 544bed47b1300f1162a2ed59fbd16af3da780136bab72e059ea81d1d0be149798a5800af7b60811219c93bb52d6c89ea9000943f3efb46da5a1c51a668591c8f:47e3098d4412bd61704d064a00a7b4fab8bc37390d0c423f90d5e043df395ea7cb69fbf4645312252b0c83a3c02d28e0823ab44760cc6e130121199be7ebf86d 731caaaa977ec88f8c39a358879747227698c50fd60802506b9666936fd4029fc69a5afdefef754d6faa49b8f9731ab25393d2817277570efb23d13525937d8c:30379336403acdb6817d7d12f7eb0191ebaf395a8cfec8bd963fd01cac861be0685dfb1b23174e8dce2db596d745b7e19e9132e9871ace51553303f59bd5fcd1 5cfc045c6d12b0f11134f68baae887086bd9c3019f9d6dc3a74a1f26b9e895716926432bd07570aa2957dfa5c53b0deaed9570e40c6446cdd98fd7155876babf:efb57b110165edc3b369e4a7c66c1d1a8e3b51c5a7d12e98c8b4c522dda6bd30d4b5ab9c0e4d44b13dcb12bbfa9a1d32233cb5f9016d03191457047461f5272a d45be497332193978fc1ff399465a4249eee3bc9a91ee40aa8f73b4a03ca11f92b2b878b05714bbfa7b676df0e6b63f1e8f7c469c9ed99770f4f51dbb599320d:4224cea2edf604cabd636667c07360c67816d9bf800fa25ad591abd887d223496f8b733548a94c31d68fcfbcac39405eb5837dcae160f3e25398f6de14d4837a f55fd0c26f686658dcfdc27b62fcc19e34f6e0da0d967431525d3b24eecbeb1f97d499704f035f45be48949aed04ed82b15e694c6416f0ff5a8a938341fc95d4:9a84259756300a7ce8ea8b60d1619096d0f78f2950ca171953c1f704e40e971037fea5dfe95ccae7494e35e6be2825dba608316d00d3ec8ab79e2f0ea7f33adc d08093ae54de27423063b590fd986db0bc2feb21ea8cd9f6aa516c1a5e80b2580f93c140292f7ca270f2e8436f70749963b2efb26f07c2ab5fd2d24b6e243ede:cbf6e85e52308fdd28ab4c6d2bde03b59c3de534882c8070c08048994c1e3de47aa70ab377c67a5e0ca7bf64e46fb24e5efe73b813e721c55bbca1efc9a380fd acc8a2ee65853e29e768a3d14a4ca598615521d3ea8b1ae11d421b8403355ea11b942632d76de33d3a3923c77bba3526276afee3f3307082897c5173cf7880f6:b580d7c0ebb0e97ca2259102a1d287ae58991f6caeb3e31e7de0012cf726db3f5746184e0407a1f55779fea356e2e463291dcd8c85f71586b96025129b47e182 9431232f4e2b40b8a06fab7207ab45a51bd43f3d00fb546e99fdeda8408ff4458a78ac9658b7b8ee526fefdb3d367f49a5504d21beb909e25b10171067d8dbd8:a110b74388a6b5378e7837de2cc9184d0a4887ee373a2297c30015032de3db30f32a01e56b8faa60111620d4483226a2e7008c09a17f99ff47beb9f03dc2c2fd 1d6bc8a35bbcde6b3a5a5e7e67690a2ab882fc1d1c1a4657506f1d8237d1de5c28243e6fbe3d8b72f2fbb63340fa73f6d96db7155ae89b82fec638b47cf6eb6a:1db82c674c9cb19090166765a92e56c002e7509e619acdbb4d046532a566b25942bbfc6f5380ce40cdd157fd6ffca76aabca7197b11afcd75cc82cc173dbedb5 c89a35833525e3715e94754736bd7b64f2d5ede02491e24394e71863f3e5fdfef493e46825789c6d0c0e7129eadf92647ca29a6fc966b5faf4cb18d8c10feeab:5e817e12a4833e1a4318bbacbc2775f5aae7e640d5dd62eac65015006eb217ed9ce4c911db382b45f01285dd909684a4c8051d000d820cc3a31263f5e6223e7f 07ff4c5d0cfc1229e2ea4d817cf3b7adde528ff27c014ef51715dbab76573c84baedcbb2c3b0a042f7c5779f30d09f1a753ba57a10908f6ae03df66abe89abf0:fec16d6fe85458111a50fff95cfdbe9872b608aa6973df03035d8fd17801656743bdac7a120d27d1dc60dc9e7785d220ad3e7a6f40efe3248d11be485a9c5431 eb2eacceafe92180ab1ce9597bb15280039e3f2241eaf8a67d4ccd2f6383bbd46894304f3ffc7592cad36fb21965f1bd8c7e6a8cfb3f643c8eb0fe841b919086:311be4612a2d1eb8fda7f61c983940369908896896df727e71d041ebe383fc77e40310f302667ddce20123f5b1c88628cc61bbac44db507342782b658312dd0b 7682bb90f64ff99363b0646bdd913680ac48347f18fd24612f7e409712dbea4e25ceb159451117cf4d7c8bdf006d6abb8062f480aab1e2d9dd5f4a9940c7ed29:653cbcce3f5947d678c72146d9e9899e9bd865b7cf2c5fb4edb8443880c702066f5360f9dcd7b5e4344eb2f2579e6268805389cfcac0845b07858ff74faf734b 6f4b56761ea58672c11023a8b0848eb7b53b22308801ba48eeb4f8293f4e7864f32256179775c648fd400e3e857a03e2ed92839aa95c0e7b9afee4badfa7ab48:00f965e54d65916d1a4079dcc1ba7b7bbe413e413046c616f66df8638f146c624b2e3d7960dd71875f24758196e352b8677f1fb336af0d77d7bd4490125c107a fe092faf9a3bfb62e1b37081a292303dae1965f3ae2d43a0ac7665ccefcc4fb4ce4313b964f96046d9f6147c9e3c12e28638c8fce4aa09ca7e2f5bbb49f341f0:851cf1d771c2e1ca913c97343c8f3dd27de6b41067008640da70960ea5fb1ba909a10e688761d75cfc60a71ba94ba74afb556be2fde2149dd17883738ccf364e d78ca3a1ddc6a599307484e278bcfe8a3b10d55bb94e7670e7096f1169df1cea9825869e43da27a3f7262bd863ec3a82927ca154240b53b4c8f8b6e2fe69b5f3:8c91987e236274eed5957d0ea453764d8f9a3d111591ee9b7118ae31ed891c20163b6240b7859613e62933568d75605ce49b6e17b2f772e5178344e4c7f66985 c3c3ec7a7d9e9126233e498e092776bace142f66219e5e89472cfe8222e51d1331a6294a68d75eebab1035b0e02ed69ec9451712fc3c3e20dd6320b11618811f:966f4fb9cd80af0aa108df47da1271137ebe81a3f7282518063b2d241ab3588d00c2141c8e39a795a12d23a3a0732b3c6b3de1274e3630d6ec7571a2ef8c7afdMost n00bs would stop right there, because wtf amirite? But you've got mad skillz, and access to the source code (databases are harder to get into than the server itself, so it is not unreasonable to assume an attacker can read your source code), so you can build a dictionary to test these values against. Go ahead, I'll wait. Each password has it's own salt, which means you'd need to re-build your rainbow table for *every* user. Which would take roughly 3 days for a password 5 characters long.... per user (for sha512. Using the proper hash algo, it'll take significantly longer. Longer than "heat death of the universe" kind of significant).
md5 and sha512 aren't actually good enough for passwords for a variety of reasons. They're too fast. Well, and md5 is broken (two different passwords can yield the same hash). Time is the enemy. If you give an attacker enough time, they can eventually unwind whatever insane algorithm you're using to encrypt your password, and then apply the same process to a rainbow table, then compare it with your database to get the passwords. So the longer something takes, the better it is. sha512 and md5 are designed to be fast.
Luckily, this is 2016 (almost 2017!). We have more than just some random loon proclaiming things are bad and other things are good. We have many loons. Science has come to the rescue, and delivered upon you a simple solution: use bcrypt. It's been tested hard for years, it's designed for passwords, it's slow on-purpose, and it's available for almost all languages. In python, it's as simple as "pip install bcrypt".
>>> import bcrypt >>> user_password = "spam spam eggs and spam" >>> hashed = bcrypt.hashpw(user_password.encode(), bcrypt.gensalt()) >>> # this is what you store in your database. ... >>> hashed b'$2b$12$fZLMiMQrovVwRQt9vO7hW.6FNW1NU/1LIYVLpG1LGaT7kWAmlBDy2' >>> # later... ... >>> bcrypt.hashpw(user_password.encode(), hashed) == hashed TrueThat's very simple, easy to understand, and almost guaranteed to be more secure than anything you can dream up on your own. Use it. Embrace it. Love it.