** Python Support Thread ** - Page 18 - Computer Technical Help

Two Plus Two Forums Other Topics Computer and Technical Help

** Python Support Thread **

Post Reply Subscribe

...

Page 18 of 30

First

8 13 14 15 16 17 18 19 20 21 22 23 28

Last

Page 18 of 30

First

8 13 14 15 16 17 18 19 20 21 22 23 28

Last

01-18-2012 , 12:54 AM

#426

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

Being able to use functions as arguments and return values is like in the top three reasons if not the #1 reason I use Python

Quote

150% up to $2,000 Welcome Bonus on CoinPoker

Join the action now

Daily Rewards • Splash Pots • CoinRaces

01-19-2012 , 03:10 AM

#427

daveT

S.A.G.E. Master

Join Date: Jun 2005 Posts: 23,955

Quote:

Originally Posted by Xhad

Being able to use functions as arguments and return values is like in the top three reasons if not the #1 reason I use Python

If you like this property, I strongly suggest learning some Lisp. You can go nuts with this concept.

I also didn't know this property was so unusual that it is a great reason to like Python. I mean, Javascript, Ruby, Haskell, et. al. ftw?

Since I'm not sure if you were referring to my post, I wanted to clarify that I was talking specifically about closures, which are a massive mind-**** at first. I feel like my single biggest break-through in programming to date was identifying a situation where a closure was reasonable, and then actually getting the thing to work correctly. Of course, this doesn't mean I can explain it to a six year old, but it means I can identify what it does.

Basically, a closure function holds a value so you can call it again later with other arguments. Instead of a mutable variable, you get a sort of mutable state of a function.

http://stackoverflow.com/questions/2...to-use-closure

I think the MDN docs give the best explanation I have seen, though it is in javascript.

https://developer.mozilla.org/en/Jav...Guide/Closures

Quote

01-19-2012 , 10:45 AM

#428

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

Well then maybe I'm unlucky in that the languages I've worked with the most in any "official" capacity happen to be the ones that don't allow it (like C).

Lisp-like languages are interesting but run afoul of some of my other necessities like having good libraries (I'm not sure I would have picked up Python if it weren't for numpy).

Quote

01-19-2012 , 11:09 AM

#429

tyler_cracker

Carpal \'Tunnel

Join Date: Apr 2005 Posts: 15,828

a buddy of mind spoke highly of Clojure since you get all the power of lisp plus it's super easy to integrate with any Java library you like (and, as you may be aware, there are a few of those in the wild).

Quote

01-20-2012 , 01:23 PM

#430

mmbt0ne

Carpal \'Tunnel

Join Date: Aug 2004 Posts: 22,077

I've got some ugly (working!) code that I want to improve.

First, an explanation of what's going on here. This is an email copy-pasted from what I sent to friend of mine:

Quote:

So, I'm in this weird spot right now where the script actually does more than I originally planned, but only has the extra logic half-finished.

Basically what it does is:
1) Copy any known numbers from the starting matrix to the answer matrix
2) Look at a cell, and then create an array of any number it CAN'T be (already in the row, column, or square)
3) Remove any number in the "can't-be-the-answer" array from the answer array
4) Do this for all 81 cells
5) Any answer array that has a length of 1, convert to an integer
6) If the answer matrix has changed since starting step 2, do this loop (steps 2 through 5) again
** This works for all easy sudoku puzzles **
** If you just end now when the answer doesn't change, you're good to go **
** Here is where **** gets shady **
7) If the answer hasn't changed, then look at the first row
8) Figure out what the "missing numbers" are in that row
9) If there is a missing number that is only an option in one cell, put it there
10) Do this for all 9 rows
11) If the answer matrix has changed, go back to step 2.
12) If the answer matrix hasn't changed, print the answer matrix

The piece that's missing is doing that same pass9 function for columns and squares like I do for rows. It wouldn't be THAT hard to implement, but I have to think about exactly how I'd do it.

This code is fairly easy to read but probably an abortion to anyone who writes in Python a lot. Thankfully I don't think you qualify yet.

The example in here was from the "Hard" level of my sudoku app. I had already beaten "Easy" and "Medium" with steps 1-6. It took 19 iterations to complete and basically no time except for the printing functionality

And now, the good stuff! Where am I doing things "wrong" that can either a) make it more efficient or b) make it more Pythonic and easier to read.

Code:

# My attempt to solve a basic sudoku puzzle

# This is the starting point
# Blank spaces are represented by 0's
start = [[4,0,0,5,0,0,0,1,0],
         [0,3,0,0,0,0,0,0,8],
         [0,0,0,0,2,0,3,0,0],
         [0,0,4,0,0,9,5,0,2],
         [2,0,0,0,0,0,0,0,6],
         [5,0,0,0,0,0,4,3,0],
         [0,0,7,0,0,3,0,0,9],
         [0,0,1,9,6,0,0,8,0],
         [6,0,0,0,1,7,0,4,0]]

# Big, long, ugly answer matrix
ans = [[[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]],
            [[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]],
            [[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]],
            [[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]],
            [[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]],
            [[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]],
            [[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]],
            [[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]],
            [[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],
            [1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9],[1,2,3,4,5,6,7,8,9]]]

changed = True

# Remove numbers that also appear in
# a direct row, column, or square
def rowcolcheck(chk, answ, r, c):
    for numb in chk:
        if answ.__contains__(numb):
            answ.remove(numb)
    ans[r][c] = answ

# If the answer is down to a list of length 1
# then just copy it out of the list to a normal number
def makereal(answ):
    for row in range(9):
        for column in range(9):
            if type(answ[row][column]) == list and len(answ[row][column]) == 1:
                ans[row][column] = answ[row][column][0]

# Defines the 3x3 squares
def defineSquare(r,c):
    numbersUsed = []
    rows = []
    columns = []
    
    if r < 3: rows = [0,1,2]
    elif r < 6: rows = [3,4,5]
    else: rows = [6,7,8]

    if c < 3: columns = [0,1,2]
    elif c < 6: columns = [3,4,5]
    else: columns = [6,7,8]

    for row in rows:
        for column in columns:
            if type(ans[row][column]) != list: numbersUsed.append(ans[row][column])

    return numbersUsed

# Take in 9 sudoku spots in a row
# If one of the spots is the only option
# for a given value, put that value there
def pass9 (listRow,ansRow):
    possibles = []
    yesses = []
    
    for entry in listRow:
       if type(entry) == list:
           for subEntry in entry:
               possibles.append(subEntry)
               
    for n in range(1,10):
        if possibles.count(n) == 1:
            yesses.append(n)

    for z in range(9):
        if type(listRow[z]) == list:
            for yes in yesses:
                if listRow[z].__contains__(yes): ans[ansRow][z] = yes


# Copy the existing answers
for row in range(9):
    for column in range(9):
        if start[row][column] != 0:
            ans[row][column] = start[row][column]

x=1
while (changed):
    otherans = []
    print "iteration", x
    x += 1
    changed = False

    for y in ans:
        otherans.append(y[:])

    # Cancel out rows and columns
    for row in range(9):
        for column in range(9):
            if type(ans[row][column]) != list: continue
            checker = []
            for point in ans[row]:
                if type(point) != list: checker.append(point)
            for point2 in ans:
                if type(point2): checker.append(point2[column])
            for point3 in defineSquare(row,column):
                checker.append(point3)
            rowcolcheck(checker, ans[row][column], row, column)

    makereal(ans)

    if otherans != ans: changed = True
    else:
        for pt in range(9):
            pass9(ans[pt],pt)

        makereal(ans)

        if otherans != ans:
            changed = True


for row in range(9):
    for column in range(9):
        print ans[row][column],
    print

Quote

01-20-2012 , 01:43 PM

#431

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

Will edit this post as I find stuff

Ugly ans matrix made less ugly:

Code:

ans = [[[i for i in range(1,10)] for j in range(9)] for k in range(9)]

In rowcolcheck: Don't use __contains__. Use "in":

Code:

if numb in answ:

But then I want to replace that entire loop with:

Code:

answ = [x for x in answ if x not in chk]

I'll have to look again after lunch because I'm not sure what all the code does yet, but I will say all the type(whatever) == list looks like it's probably unnecessary. I'd be more inclined to just make sure everything is always a list (or possibly set) and check len(x) == 1, if that much is even necessary.

Last edited by Xhad; 01-20-2012 at 01:52 PM.

Quote

01-20-2012 , 02:05 PM

#432

mmbt0ne

Carpal \'Tunnel

Join Date: Aug 2004 Posts: 22,077

Thanks Xhad!

Quote:

In rowcolcheck: Don't use __contains__. Use "in":

Can you explain this? Not as in "How does it work" but more "why is it better?"

Quote:

Code:

answ = [x for x in answ if x not in chk]

Ah yes, I was trying to remember this because I know I've rewritten some code a while back to fit this type of formatting. What's it called again?

Quote

01-20-2012 , 02:15 PM

#433

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

This is hard to google for obvious reasons, but I'm pretty sure "in" just calls "__contains__" (except for dictionaries and custom classes where __contains__ isn't defined and it has to call something else instead). So it's just as fast but more readable and slightly more flexible.

That construct is called a "comprehension".

Quote

01-20-2012 , 11:09 PM

#434

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

So, running the thing on my machine now and playing with it. This caught my eye in defineSquare:

Code:

if r < 3: rows = [0,1,2]
    elif r < 6: rows = [3,4,5]
    else: rows = [6,7,8]

    if c < 3: columns = [0,1,2]
    elif c < 6: columns = [3,4,5]
    else: columns = [6,7,8]

Because I immediately realized this shorter version:

Code:

r0, c0 = r//3*3, c//3*3
rows = [r0, r0+1, r0+2]
columns = [c0, c0+1, c0+2]

I was wondering if that's actually less readable and therefore not worth it, and if a comprehension would be too cute there, until I realize that literally the only thing you do with these lists is use them as for loop indices, so the whole function became:

Code:

def defineSquare(r,c):
    numbersUsed = []

    r0, c0 = r//3*3, c//3*3

    for row in range(r0, r0+3):
        for column in range(c0, c0+3):
            if type(ans[row][column]) != list:
                numbersUsed.append(ans[row][column])

    return numbersUsed

A note about the '//', I use Python 3. Since you use Python 2 you'd actually use just a single slash. This also means that you might want to get into the habit of typing xrange instead of range for things like loops because the performance is better (not by a huge amount for lists this small, but something worth keeping in mind)

I also changed building "otherans" into a one-liner by importing copy.deepcopy, but now I'm looking for a way to dispense with it completely. I really want to believe there is a way to keep track of whether ans has changed without using "==" on two multidimensional lists.

EDIT: My reaction to this if condition was "Is this on purpose?"

Code:

for point2 in ans:
                if type(point2): checker.append(point2[column])

The thing appears to work, but I don't quite get why.

Last edited by Xhad; 01-20-2012 at 11:26 PM.

Quote

01-21-2012 , 01:59 PM

#435

mmbt0ne

Carpal \'Tunnel

Join Date: Aug 2004 Posts: 22,077

Quote:

Originally Posted by Xhad

So, running the thing on my machine now and playing with it. This caught my eye in defineSquare:

Code:

if r < 3: rows = [0,1,2]
    elif r < 6: rows = [3,4,5]
    else: rows = [6,7,8]

    if c < 3: columns = [0,1,2]
    elif c < 6: columns = [3,4,5]
    else: columns = [6,7,8]

Because I immediately realized this shorter version:

Code:

r0, c0 = r//3*3, c//3*3
rows = [r0, r0+1, r0+2]
columns = [c0, c0+1, c0+2]

Code:

def defineSquare(r,c):
    numbersUsed = []

    r0, c0 = r//3*3, c//3*3

    for row in range(r0, r0+3):
        for column in range(c0, c0+3):
            if type(ans[row][column]) != list:
                numbersUsed.append(ans[row][column])

    return numbersUsed

This is awesome, but can you explain what's going on exactly? I mean, I can tell that it's ending up with the same groupings, but I don't really understand

Code:

r0, c0 = r//3*3, c//3*3

Quote:

Ok, I remember reading about xrange stuff a while back when I was going through some of the free books out there on Python. Thanks, I'll switch that stuff over.

Quote:

EDIT: My reaction to this if condition was "Is this on purpose?"

Code:

for point2 in ans:
                if type(point2): checker.append(point2[column])

The thing appears to work, but I don't quite get why.

No, it's definitely NOT on purpose. I am also not sure why it works, but I do know that this is almost 100% the reason I kept getting errors when I tried to put the pass9 code inside the normal loop. I kept erroring out after a few iterations because an int would sneak into the array some how, and while I haven't traced it back I would have to guess this is the genesis since I'm not actually filtering here like I thought I would be.

Quote

01-21-2012 , 02:05 PM

#436

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

Quote:

Originally Posted by mmbt0ne

This is awesome, but can you explain what's going on exactly? I mean, I can tell that it's ending up with the same groupings, but I don't really understand

Code:

r0, c0 = r//3*3, c//3*3

In Python 2 it would just be:

Code:

r0, c0 = r/3*3, c/3*3

The reason it works is because it's integer division (which rounds down), so it's essentially "highest multiple of 3 that is less than or equal to r (and c)" Maybe this form makes more sense:

Code:

r0, c0 = r - (r%3), c-(c%3)

Quote

01-21-2012 , 09:52 PM

#437

e i pi

veteran

Join Date: Feb 2010 Posts: 2,230

any of you guys do any web scraping with python? it seems like the most popular packages for scraping (beautifulsoup) aren't so great or just don't exist for python 3. I tried the beta for beautifulsoup and got errors when trying to import the module

Why haven't they kept the libraries up to date with python 3? Can I have both python 2 and 3 on my computer at the same time? One of my main motivations for learning to program in the first place was so I could scrape sites.

Quote

01-21-2012 , 10:48 PM

#438

tyler_cracker

Carpal \'Tunnel

Join Date: Apr 2005 Posts: 15,828

Quote:

Originally Posted by e i pi

Why haven't they kept the libraries up to date with python 3?

probably because 1. lack of resources to do the port and 2. lack of demand. you could address #2 by finding the appropriate place in the project and filing a request. you could address #1 by digging into those errors you're seeing and contributing fixes back to the project.

Quote:

Can I have both python 2 and 3 on my computer at the same time?

sure.

Quote

01-22-2012 , 12:10 AM

#439

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

Also, utilities exist to convert python2 code to python3 and any decent library developer certainly knows this, so whatever python2-only libraries are still around are most likely difficult or at least nontrivial to port.

Quote

01-22-2012 , 12:31 AM

#440

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

So on the Sudoku thing, I think I've gotten to the point where any more playing with it will be for my benefit rather than mmbt0ne's. I do want to mention a couple more things though:

I finally realized an easy way to get rid of that "otherans" thing that's bothering me: make an AnswerMatrix class like so:

Code:

class AnswerMatrix:
    def __init__(self):
        self.ans = [[[i for i in range(1,10)] for j in range(9)] for k in range(9)]
        self.changed = True

And pass the class around to your other functions. Any function that changes ans should also set the class's "changed" variable to True. That should save some resources over repeatedly copying ans and using == to compare to old copies of itself. (I suppose the "purer" OOP version of this is modifying ans through member functions and having those member functions set the changed flag, but that would require more work)

Second thing, still don't know enough to know if this is easily possible (or if it would add that much in performance even in larger puzzles), but I really suspect ans could use sets instead of lists. They're often faster (since they're hash tables rather than arrays) and allow operations that make certain things easier. For instance, that answ thing from earlier? If they were sets and not lists this actually works:

Code:

answ -= chk

Btw, it would be a lot easier to try if not for the constant type checking against list, since I don't think you ever use any list-specific methods like .append on the ans matrix (len works on sets too). In my reading I've noticed there's definitely a school of thought that regards excessive type checking as a bad habit for this exact reason.

Quote

01-22-2012 , 01:03 AM

#441

Neko

Pooh-Bah

Join Date: Jul 2005 Posts: 4,000

Code:

[i for i in range(1,10)]

can be more concisely written as

Code:

range(1,10)

and yeah using isinstance or type to do type checking is a bit of a code smell and usually frowned upon.

Quote

01-25-2012 , 11:33 PM

#442

Xhad

Carpal \'Tunnel

Join Date: Jul 2005 Posts: 10,962

So, yet another IDE I just found out about:

http://idlex.****************/

Essentially, it's a Python script that runs IDLE but with a lot of the suck patched out of it. I've been running it for "desktop calculator mode" for the last few days and I've definitely noticed a distinct lack of things like crashing and spawning processes for no reason.

Quote

01-27-2012 , 12:26 AM

#443

daveT

S.A.G.E. Master

Join Date: Jun 2005 Posts: 23,955

Quote:

Originally Posted by e i pi

I have both 2 and 3 on my computer. I just downloaded Scrapey yesterday, which works only on Python 2. Not sure how this one compares to any of the others, but it is pretty minimalist and robust at the same time if that makes any sense at all. I fully expected that any open source scraper is going to require tons of your own coding to make it work any good.

Quote

02-04-2012 , 02:24 PM

#444

e i pi

veteran

Join Date: Feb 2010 Posts: 2,230

Thanks! I just got scrapy installed yesterday and am getting acquainted with it. It does seem like a lot of work but there are some examples and open source code for it.

https://github.com/scrapy/dirbot
http://snippets.scrapy.org/

bwaha i'll be working for google in no time

Quote

02-22-2012 , 11:03 PM

#445

Sholar

Carpal \'Tunnel

Join Date: Jul 2007 Posts: 6,357

I'm trying to get 3to2 to work on a windows machine, with zero success. Is there a simple step-by-step instruction set somewhere?

I made the choice to start with Python3 a few months ago, and am now there are some packages I'd like to use that haven't been ported yet. It's mostly little things that I'll need to change (printing, division, csv module parameters, etc.) but I assume 3to2 will do all of that by magic once I have it working rather than me, more tediously, trying to make these changes by hand.

So I've downloaded 3to2 but if I run "python3.2 setup.py install" it generates a bunch of syntax errors, so I must be missing something...any ideas?

Quote

02-22-2012 , 11:08 PM

#446

Neko

Pooh-Bah

Join Date: Jul 2005 Posts: 4,000

3to2 should be run with python2.x rather than python 3.2 iirc.

Quote

02-22-2012 , 11:19 PM

#447

Sholar

Carpal \'Tunnel

Join Date: Jul 2007 Posts: 6,357

You're right, thanks! Not sure how I missed that bit of info...

Quote

02-22-2012 , 11:21 PM

#448

daveT

S.A.G.E. Master

Join Date: Jun 2005 Posts: 23,955

Quote:

Originally Posted by Sholar

Assuming you are starting new projects, why not download Python 2?

Quote

02-22-2012 , 11:35 PM

#449

Sholar

Carpal \'Tunnel

Join Date: Jul 2007 Posts: 6,357

Yeah, I'll continue things in 2.7, but the problem is that I've already written a bunch of stuff in 3.2 over the past few months, and now need to convert that. Basically just a lack of foresight on my part that I'd want to use some libraries which weren't supporting 3.x yet.

(And 3to2 sadly is not magical...it looks like a lot of the details will need to be fixed by hand anyway.)

Edit: The "UnicodeEncodeError"s are just rubbing salt into the wound here :-)

Last edited by Sholar; 02-22-2012 at 11:47 PM.

Quote

02-24-2012 , 01:30 AM

#450

daveT

S.A.G.E. Master

Join Date: Jun 2005 Posts: 23,955

Eh, I haven't tried to use 3to2 yet, but I could definitely see where many bad things could happen. I like using 3 better than 2, but oh well.

I use 3 to do database stuff and 2 to do web scraping atm. Otherwise, I do Project Eulers with 3. I guess I'm not much of a programmer with Python these days.