The Joys Of Programming

bandit0013 wrote:

I like that attitude, yet I constantly come across programmers who have absolutely no interest in understanding relational databases even though 90% of the time that's where the performance problems are because they didn't care and treated it like a flat file store.

Its funny that you mention that, as I have no interest in understanding relational databases (at least nothing more then the basics), but at the same time I would rather work closely with someone who knows relational databases inside and out rather then pretend that I know what I am doing. Maybe one day I will take the time to know more details but I am more of a fan of the NoSQL databases that are becoming popular.

Edwin wrote:

I made a tip calculator. It's pretty basic but as I learn stuff, I'll be expanding it. It's not that big of a deal, but to me it is.

Congrats! It's a good time to learn to code!

Sadly, NoSQL is still in its infancy and even the top products don't always scale well. Last I heard, MongoDB acquired a global lock for every transaction, for example. Compared to a reportable SQL database, this is laughable.

kazar wrote:
bandit0013 wrote:

I like that attitude, yet I constantly come across programmers who have absolutely no interest in understanding relational databases even though 90% of the time that's where the performance problems are because they didn't care and treated it like a flat file store.

Its funny that you mention that, as I have no interest in understanding relational databases (at least nothing more then the basics), but at the same time I would rather work closely with someone who knows relational databases inside and out rather then pretend that I know what I am doing. Maybe one day I will take the time to know more details but I am more of a fan of the NoSQL databases that are becoming popular.

I'm a fan of NoSQL too, but I strongly urge you to learn and appreciate the fundamentals of relational databases. It is an excellent way to learn how to design for data integrity and scalability, store untrusted input (NoSQL doesn't mean no SQLi-type attacks!), correctly perform operations using the relational algebra that most query systems derive from, take advantage of precomputed data representations and statements, and -- perhaps most importantly -- troubleshoot common problems that occur on systems used by the vast majority of the programming world related to encoding, truncation, bad indexes or foreign keys, stupid MySQL default type conversions, and so forth.

There are use cases where I love NoSQL; I've used MongoDB for storing denormalized user profiles for easy access, am using HBase to store sparse representations of feature vectors for my machine learning framework in an intuitive way, and if you tune for high write throughput, sometimes it's a good choice for logging. But a lot of the smarter companies, in my opinion, seem to be selectively using NoSQL only for certain applications in order to leverage the awesomeness of traditional RDBMS to the fullest extent.

Lots of people suggest MySQL for learning, because it's cheap and easy and common... I would recommend that you check out PostgreSQL as it seems to be having a resurgence in popularity and has been getting more and more nifty features (lots of NoSQL lovers are impressed with the PostgreSQL hstore, I hear).

ADD: Also, understanding traditional RDBMS is valuable for being able to critique technologies which are competing with NoSQL for the future of data management (usually termed "NewSQL").

complexmath wrote:

Sadly, NoSQL is still in its infancy and even the top products don't always scale well. Last I heard, MongoDB acquired a global lock for every transaction, for example. Compared to a reportable SQL database, this is laughable.

You last heard a while ago! That was an acknowledged defect and they've made some good strides on it.

There are definitely caveats to the newer systems (for the love of Pete, if you use MongoDB, make sure you have enough memory to store your working set and indexes!)... but that doesn't mean they're not worthwhile in the right case. Always match the technology to the use case.

EDIT: I hear Riak is the new hotness, but I haven't tinkered with it yet. Has anyone here?

kazar wrote:

Maybe one day I will take the time to know more details but I am more of a fan of the NoSQL databases that are becoming popular.

I never really understand these kinds of statements, relational databases and NoSQL systems solve such fundamentally different problems that I don't really see that there is a sensible comparison to be made. You're not going to use a NoSQL system for a problem that can be best addressed with structured, normalised data and you probably shouldn't be using a relational database to index your petabyte document store.

DanB wrote:

I never really understand these kinds of statements, relational databases and NoSQL systems solve such fundamentally different problems that I don't really see that there is a sensible comparison to be made. You're not going to use a NoSQL system for a problem that can be best addressed with structured, normalised data and you probably shouldn't be using a relational database to index your petabyte document store.

I have been finding that more often then not, relational databases are not the right answer, they are the default answer. I have seen too many projects where the first step of design is "what should the database schema look like" instead of "what should the object model look like". I prefer to spend most of my time on the object model and business logic and leave the database side to an expert. I prefer NoSQL solutions because it leaves more of the relational side of things to the app developer instead of hiding it in complex relational databases.

Cyranix wrote:

I'm a fan of NoSQL too, but I strongly urge you to learn and appreciate the fundamentals of relational databases. It is an excellent way to learn how to design for data integrity and scalability, store untrusted input (NoSQL doesn't mean no SQLi-type attacks!), correctly perform operations using the relational algebra that most query systems derive from, take advantage of precomputed data representations and statements, and -- perhaps most importantly -- troubleshoot common problems that occur on systems used by the vast majority of the programming world related to encoding, truncation, bad indexes or foreign keys, stupid MySQL default type conversions, and so forth.

There are use cases where I love NoSQL; I've used MongoDB for storing denormalized user profiles for easy access, am using HBase to store sparse representations of feature vectors for my machine learning framework in an intuitive way, and if you tune for high write throughput, sometimes it's a good choice for logging. But a lot of the smarter companies, in my opinion, seem to be selectively using NoSQL only for certain applications in order to leverage the awesomeness of traditional RDBMS to the fullest extent.

Lots of people suggest MySQL for learning, because it's cheap and easy and common... I would recommend that you check out PostgreSQL as it seems to be having a resurgence in popularity and has been getting more and more nifty features (lots of NoSQL lovers are impressed with the PostgreSQL hstore, I hear).

ADD: Also, understanding traditional RDBMS is valuable for being able to critique technologies which are competing with NoSQL for the future of data management (usually termed "NewSQL").

I have the "fundamentals of relational databases" pretty much covered. I don't think any developer could do their job without running into an SQL database of some kind and have to learn the basics. But that being said, it would be dumb for someone who just has the fundamentals covered to be designing DB schema, writing SQL statements that will be used in productions, etc... but I see this happening every day. Even with just the fundamentals, I can tell when the DB schema I am working on top of is poorly done.

Edwin wrote:

I made a tip calculator. It's pretty basic but as I learn stuff, I'll be expanding it. It's not that big of a deal, but to me it is.

I love how excited you are. It's a really good sign.

No need to use the + there for the output edwin, just "a string of stuff %.2f" % float_value.

kazar wrote:
DanB wrote:

I never really understand these kinds of statements, relational databases and NoSQL systems solve such fundamentally different problems that I don't really see that there is a sensible comparison to be made. You're not going to use a NoSQL system for a problem that can be best addressed with structured, normalised data and you probably shouldn't be using a relational database to index your petabyte document store.

I have been finding that more often then not, relational databases are not the right answer, they are the default answer. I have seen too many projects where the first step of design is "what should the database schema look like" instead of "what should the object model look like". I prefer to spend most of my time on the object model and business logic and leave the database side to an expert. I prefer NoSQL solutions because it leaves more of the relational side of things to the app developer instead of hiding it in complex relational databases.

Right. I can totally get that. I think that 'default answer' issue comes down to people not really considering or genuinely appreciating the structure or relationships present in their data.

DanB wrote:

I never really understand these kinds of statements, relational databases and NoSQL systems solve such fundamentally different problems that I don't really see that there is a sensible comparison to be made. You're not going to use a NoSQL system for a problem that can be best addressed with structured, normalised data and you probably shouldn't be using a relational database to index your petabyte document store.

Some of this is definitely the relative maturity of SQL vs. NoSQL systems. That said, something like a document store could use an SQL database for the index and metadata with the actual files stored on a filer or HTTP server somewhere. Searching in a NoSQL system still seems a bit weak compared to searching in SQL. For pure key/value retrieval though, NoSQL is great.

complexmath wrote:
DanB wrote:

I never really understand these kinds of statements, relational databases and NoSQL systems solve such fundamentally different problems that I don't really see that there is a sensible comparison to be made. You're not going to use a NoSQL system for a problem that can be best addressed with structured, normalised data and you probably shouldn't be using a relational database to index your petabyte document store.

Some of this is definitely the relative maturity of SQL vs. NoSQL systems. That said, something like a document store could use an SQL database for the index and metadata with the actual files stored on a filer or HTTP server somewhere. Searching in a NoSQL system still seems a bit weak compared to searching in SQL. For pure key/value retrieval though, NoSQL is great.

Searching across items in NoSQL is awful... because that's not what it is for. If you want relational aggregation, reporting, ACID, etc then a traditional RDMS is the way to go. If you are almost always working with single records in a key/value type store the NoSQL is blazingly fast and efficient.

kazar wrote:
SixteenBlue wrote:

Sure, there are definitely multiple traits I'd look for in a potential hire. I just meant if all other things are equal, I'm taking the person who leverages existing solutions over the person who rolls their own.

Of course there are times when rolling your own is the best answer, but the person who always does it is not really in a good position to explain why.

A person that rolls his own solution will have a better understanding of how things work. The industry is flooded with people that just know frameworks but have absolutely no clue how those frameworks work. I am a firm believer that people should know how to do things from first principles.

While in the end, leverageing existing solutions has huge advantages such as the code being well tested and the time saved in development. So I am not advocating that a person should roll their own, but I would rather have someone who would normally roll their own and teach them how to leverage existing solutions vs the other way around.

I forgot to respond to this, sorry about that.

The bolded part might be true but it's not always. I do agree with the general concept of understanding first principles, but I'm not a roll my own kind of guy but when I use a framework I like to know how it actually works and not be the annoying douchebag who says "automagical."

Point taken on the second bolded part, assuming that person doesn't have a serious case of "Not invented here" syndrome.

SixteenBlue wrote:
kazar wrote:
SixteenBlue wrote:

Sure, there are definitely multiple traits I'd look for in a potential hire. I just meant if all other things are equal, I'm taking the person who leverages existing solutions over the person who rolls their own.

Of course there are times when rolling your own is the best answer, but the person who always does it is not really in a good position to explain why.

A person that rolls his own solution will have a better understanding of how things work. The industry is flooded with people that just know frameworks but have absolutely no clue how those frameworks work. I am a firm believer that people should know how to do things from first principles.

While in the end, leverageing existing solutions has huge advantages such as the code being well tested and the time saved in development. So I am not advocating that a person should roll their own, but I would rather have someone who would normally roll their own and teach them how to leverage existing solutions vs the other way around.

I forgot to respond to this, sorry about that.

The bolded part might be true but it's not always. I do agree with the general concept of understanding first principles, but I'm not a roll my own kind of guy but when I use a framework I like to know how it actually works and not be the annoying douchebag who says "automagical."

Point taken on the second bolded part, assuming that person doesn't have a serious case of "Not invented here" syndrome.

I agree that it isn't always true, but I would argue it is more true then the converse. When it comes to leveraging exisiting solutions, I am 100% behind you that one should do this whenever it is possible. Though I have seen this go overboard where a 1mb depenency was added just to use one class, only one method of the class and it was something simple like a string comparison function.

Pragmatic Programmers have a decent sale, thinking about diving into a couple titles myself.

40% Command-Line Special

Three great titles, now on sale, in all available formats:

tmux
Practical Vim
Build Awesome Command-Line Apps

--

Guglielmo Marconi began the first commercial transatlantic wireless service between Canada and Ireland on this day in 1907. So to celebrate the joys of powerful, unadorned technology, we’ve got three titles on sale with the same streamlined, no-nonsense philosophy. Use coupon code CommandLineFTW and save 40%.

The fine print:

valid until Wednesday, October 24, 2012.
cannot be applied to existing orders
cannot be combined with any other coupon
you have to enter the coupon code CommandLineFTW when checking out. After you’ve entered all your information, in the Order Summary, click on Apply Coupon to enter the coupon code.

Link to actual newsletter.

I'm using my monthly training materials budget to order all three for our ebook library. Thanks for the tip.

I learned conditions and control flow! To celebreate, I made a Pig Latin translator. I call it PygLatin.

Edwin wrote:

I learned conditions and control flow! To celebreate, I made a Pig Latin translator. I call it PygLatin. :o

that.... is.... awesome... thanks I needed to smile today.

Left you a comment on that file, Edwin. Your excitement is contagious!

Cyranix wrote:

Left you a comment on that file, Edwin. Your excitement is contagious! :D

That's solid advice. I really need to get my regex skills back because I never use them and I really should. I would've ended up doing something clunky like make a list of vowels and then check if the list contains first.

I can't recommend regular-expressions.info enough. Use the "Reference" section if you just need to remember something quick, but the "Tutorial" section has the best explanations. The author makes a solid effort to document features that behave differently or might be unavailable in your language of choice, which is super helpful.

I think my favorite advanced feature of regular expressions is lookaround. Finding "bar" only when not preceded by "foo": ((?<!foo)bar) -- quite handy in certain situations!

Cyranix wrote:

Left you a comment on that file, Edwin. Your excitement is contagious! :D

Thanks! I took you're advice and did a little update on it based on what I learned today.

Before:

if first == "a" or first == "e" or first == "i" or first == "o" or first == "u":

to after:

if first in "aeiou" == True:

just

if first in "aeiou":

Note that you can continue the == True forever. No need for even 1

if first in "aeiou" == True == True == True == True == True == True:

is even the same thing.
Dont do

if boolean == True:

Sorry man its a pet peeve.

boogle wrote:

just

if first in "aeiou":

Note that you can continue the == True forever. No need for even 1

if first in "aeiou" == True == True == True == True == True == True:

is even the same thing.
Dont do

if boolean == True:

Sorry man its a pet peeve.

Thoughts on using == false instead of ! ?

Thanks for the correction.

SixteenBlue wrote:
boogle wrote:

just

if first in "aeiou":

Note that you can continue the == True forever. No need for even 1

if first in "aeiou" == True == True == True == True == True == True:

is even the same thing.
Dont do

if boolean == True:

Sorry man its a pet peeve.

Thoughts on using == false instead of ! ?

I generally use ! but it pays to consider its use carefully. Readability tends to be worse this way insofar as ! is easy to miss when scanning quickly, so clear variable names and short conditional expressions help to offset that effect. The best practices of a language can also influence your decision, e.g. strict equals (===) is preferred for serious JavaScript, IIRC.

For readability (at least python) I like

if not (a == b):

I think the not pops out more than !=.

Cyranix wrote:
SixteenBlue wrote:
boogle wrote:

just

if first in "aeiou":

Note that you can continue the == True forever. No need for even 1

if first in "aeiou" == True == True == True == True == True == True:

is even the same thing.
Dont do

if boolean == True:

Sorry man its a pet peeve.

Thoughts on using == false instead of ! ?

I generally use ! but it pays to consider its use carefully. Readability tends to be worse this way insofar as ! is easy to miss when scanning quickly, so clear variable names and short conditional expressions help to offset that effect. The best practices of a language can also influence your decision, e.g. strict equals (===) is preferred for serious JavaScript, IIRC.

I asked because I was docked a point in the Coursera Scala class for saying == false instead of !. Seemed kind of silly. I find there are cases where each one is more readable and strictly using one doesn't seem smart to me.

I made another thing! It's a bunch of functions involving DNA sequences and nucleotides. The stuff towards the end is pretty ugly, but it's all I know how to do at the moment.

Edwin wrote:

I made another thing! It's a bunch of functions involving DNA sequences and nucleotides. The stuff towards the end is pretty ugly, but it's all I know how to do at the moment.

That is very cool - I a learning how to do recursive subroutines in MIPS (Assembly) right now and it's not nearly as fun as what you are doing...