PowerShell script file for updating packages

[pnachtwey]

I have excluded some problem packages.


pip list --outdated --exclude idna --exclude mistune --exclude tomli --format=freeze --user > req0.txt
$content = Get-Content -Path 'req0.txt'
$newContent = $content -replace '==', '>='
$newContent | Set-Content -Path 'req1.txt'
pip install --user --upgrade -r req1.txt
pip list --outdated
pip check

[Gribouillis]

Why not turn this into a Python script? It could become multiplatform. Also a few comments to explain why it does what it does would be welcome.

[pnachtwey]

Why not turn this into a Python script? It could become multiplatform. Also a few comments to explain why it does what it does would be welcome.

Good idea. I will work on that at night while camping. It shouldn't take me long
Also, I am not a fan of Power Shell. Is any body?
I have a zsh script for my Mac Book Pro and a bash script for my Raspberry PI.
This is enough incentive for me to write an unified python package update script.
I will need to have an easy way of excluding the problem packages

[Gribouillis]

I will need to have an easy way of excluding the problem packages

If you mean in the pip command line, you could replace it by something like

from pathlib import Path
import subprocess as sp
import sys

with Path('req0.txt').open('w') as ofile:
    sp.run([
        sys.executable, '-m', 'pip', 'list', '--outdated', 
        '--exclude', 'idna', '--exclude', 'mistune',
        '--exclude', 'tomli', '--format=freeze', '--user'], stdout=ofile)

[pnachtwey]

I will need to have an easy way of excluding the problem packages

If you mean in the pip command line, you could replace it by something like

from pathlib import Path
import subprocess as sp
import sys

with Path('req0.txt').open('w') as ofile:
    sp.run([
        sys.executable, '-m', 'pip', 'list', '--outdated', 
        '--exclude', 'idna', '--exclude', 'mistune',
        '--exclude', 'tomli', '--format=freeze', '--user'], stdout=ofile)

Excellent! Your code is probably better than mine which isn't using the latest tricks. I will study your code and try it out.

import os
import subprocess
cmd="pip list --outdated --exclude astroid --exclude idna --exclude mistune --exclude mccabe --exclude wrapt --format=freeze --user > req0.txt"
os.system(cmd)
in_file = open("req0.txt", 'r')
out_file = open("req1.txt", 'w')
while True:
    in_line = in_file.readline()
    if not in_line:
        break
    out_line = in_line.replace("==",">=")   
    out_file.writelines(out_line)
    
in_file.close()
out_file.close()

try:
    if os.path.getsize("req1.txt") > 0:
        os,system("pip install --user --upgrade -r req1.txt")
        os.system("pip list --outdated > o.txt")
        os.system("pip check >> o.txt")
        o_file = open("o.txt","r")
        while True:
            o_line = o_file.readline()
            if not o_line:
                break
            print(o_line.strip())
        o_file.close()
    else:
        print("Nothing to update")
except OSError as e:
    print("File does not exist")

In any case. We have made improvements over the method at
https://www.activestate.com/resources/qu...-packages/

BTW, isn't there a better way of excluding packages than using --exclude package for each package?
I looked and couldn't find any.

@Gribouillis
edit, wait! I did see where you edit the lines and update the packages!
Does python have something like sed and xargs? I couldn't find that either.

I have a bash and zsh script where I use xargs. It is much cleaner than my Python script.

Should I put this in a separate thread?

pip list --outdated --format=freeze --exclude idna --exclude mistune --exclude mccabe --exclude pycurl --exclude PyGObject --exclude dbus-python --exclude pycairo | sed 's/==/>=/g'| xargs pip install --upgrade --user
pip list --outdated
pip check

[pnachtwey]

If you mean in the pip command line, you could replace it by something like

from pathlib import Path
import subprocess as sp
import sys

with Path('req0.txt').open('w') as ofile:
    sp.run([
        sys.executable, '-m', 'pip', 'list', '--outdated', 
        '--exclude', 'idna', '--exclude', 'mistune',
        '--exclude', 'tomli', '--format=freeze', '--user'], stdout=ofile)

Excellent! Your code is probably better than mine which isn't using the latest tricks. I will study your code and try it out.

import os
import subprocess
cmd="pip list --outdated --exclude astroid --exclude idna --exclude mistune --exclude mccabe --exclude wrapt --format=freeze --user > req0.txt"
os.system(cmd)
in_file = open("req0.txt", 'r')
out_file = open("req1.txt", 'w')
while True:
    in_line = in_file.readline()
    if not in_line:
        break
    out_line = in_line.replace("==",">=")   
    out_file.writelines(out_line)
    
in_file.close()
out_file.close()

try:
    if os.path.getsize("req1.txt") > 0:
        os.system("pip install --user --upgrade -r req1.txt")
        os.system("pip list --outdated > o.txt")
        os.system("pip check >> o.txt")
        o_file = open("o.txt","r")
        while True:
            o_line = o_file.readline()
            if not o_line:
                break
            print(o_line.strip())
        o_file.close()
    else:
        print("Nothing to update")
except OSError as e:
    print("File does not exist")

In any case. We have made improvements over the method at
https://www.activestate.com/resources/qu...-packages/

BTW, isn't there a better way of excluding packages than using --exclude package for each package?
I looked and couldn't find any.

@Gribouillis
edit, wait! I did see where you edit the lines and update the packages!
Does python have something like sed and xargs? I couldn't find that either.

I have a bash and zsh script where I use xargs. It is much cleaner than my Python script.

Should I put this in a separate thread?

pip list --outdated --format=freeze --exclude idna --exclude mistune --exclude mccabe --exclude pycurl --exclude PyGObject --exclude dbus-python --exclude pycairo | sed 's/==/>=/g'| xargs pip install --upgrade --user
pip list --outdated
pip check

[pnachtwey]

[quote="Gribouillis" pid='158393' dateline='1655357364']
If you mean in the pip command line, you could replace it by something like

from pathlib import Path
import subprocess as sp
import sys

with Path('req0.txt').open('w') as ofile:
    sp.run([
        sys.executable, '-m', 'pip', 'list', '--outdated', 
        '--exclude', 'idna', '--exclude', 'mistune',
        '--exclude', 'tomli', '--format=freeze', '--user'], stdout=ofile)

Excellent! Your code is probably better than mine which isn't using the latest tricks. I will study your code and try it out.

import os
import subprocess
cmd="pip list --outdated --exclude astroid --exclude idna --exclude mistune --exclude mccabe --exclude wrapt --format=freeze --user > req0.txt"
os.system(cmd)
in_file = open("req0.txt", 'r')
out_file = open("req1.txt", 'w')
while True:
    in_line = in_file.readline()
    if not in_line:
        break
    out_line = in_line.replace("==",">=")   
    out_file.writelines(out_line)
    
in_file.close()
out_file.close()

try:
    if os.path.getsize("req1.txt") > 0:
        os,system("pip install --user --upgrade -r req1.txt")
        os.system("pip list --outdated > o.txt")
        os.system("pip check >> o.txt")
        o_file = open("o.txt","r")
        while True:
            o_line = o_file.readline()
            if not o_line:
                break
            print(o_line.strip())
        o_file.close()
    else:
        print("Nothing to update")
except OSError as e:
    print("File does not exist")

In any case. We have made improvements over the method at
https://www.activestate.com/resources/qu...-packages/

BTW, isn't there a better way of excluding packages than using --exclude package for each package?
I looked and couldn't find any.

@Gribouillis
edit, wait! I did see where you edit the lines and update the packages!
Does python have something like sed and xargs? I couldn't find that either.

I have a bash and zsh script where I use xargs. It is much cleaner than my Python script.

Should I put this in a separate thread?

pip list --outdated --format=freeze --exclude idna --exclude mistune --exclude mccabe --exclude pycurl --exclude PyGObject --exclude dbus-python --exclude pycairo | sed 's/==/>=/g'| xargs pip install --upgrade --user
pip list --outdated
pip check

I just checked my python script. Now I have it in my python path so I can just type python -m pyupd to update my packages

[pnachtwey]

I ran into a problem. Some how I have a new package that shouldn't be updated. I need to add --exclude charset_normalized.
This caused an error so req0.txt was blank and nothing got updated.
Any ideas?
I didn't see where pip could generate errors.

[pnachtwey]

Everything had to change because now the outdated files must be exported as a .json file. The good part is that there is a file name, old version, new version and sdist/wheel entry for each --outdated file. I plan to upgrade each file one at a time. If it returns and error then I will install the old version so nothing is broken. This method may be slower but I don't need to worry about excluding files. I will skip the sdist file though.

Another change is that I am using Linux Mint Debian Edition. It force me to use venv. Since each directory has its own venv there isn't a need upgrade as much plus it is a pain to upgrade package in 30+ different venvs.