[ home / rules / faq ] [ overboard / sfw / alt ] [ leftypol / siberia / hobby / tech / edu / games / anime / music / draw / AKM ] [ meta / roulette ] [ cytube / wiki / git ] [ GET / ref / marx / booru / zine ]

/tech/ - Technology

"Technology reveals the active relation of man to nature" - Karl Marx
Name
Options
Subject
Comment
Flag
File
Embed
Password (For file deletion.)

Join our Matrix Chat <=> IRC: #leftypol on Rizon


File: 1676899032493.png (110.33 KB, 927x1307, shellancestry.png)

 No.18497

Post shell scripts and other small utility programs.

>Why should you write shell programs?

They allow you to do menial tasks in a matter of seconds or add quick fixes to special purpose text data formats (subtitle timing).
Most of my scripts handle filenames, to help with sorting my data collection. I already do most file operations in a terminal, so these were easy to automate.

I just wrote a downloader for a set of wordpress-based manga viewers dependent on readm.org, specifically daomanga.com:
#!/bin/sh
wget -O- $1 | fgrep '"images":' test.html | cut -d ']' -f 1 | cut -d '[' -f 3|tr , '
' | tr -d '\' | xargs wget

The first wget is needed to cope with redirections. Notice the lack of sed or any regular expressions.

My notes are written in a dialect of org markup. This script processes it into troff with ms macros. Normally it only generates line breaks, headings and the titlepage while leaving anything else intact, particularly eqn characters.
When it finds a semicolon, it applies some formatting to the rest of the line. I never use it though.
#!/usr/bin/awk -f

BEGIN {
    FS="\n";
}

/^$/ {
    print ".sp"; do getline; while($0 == "");
}

/^*/ {
    i=index($0, " ");
    if(i != 0) {
	print ".NH " i-1; print substr($0, i+1); print ".LP";
	next;
    }
}

/^#\+/ {
    i=index($0, " ");
    switch(substr($0, 3, i - 3)) {
    case "TITLE:":
	print ".TL"; print substr($0, i);
	break;
    case "AUTHOR:":
	print ".AU"; print substr($0, i); print ".AB no";
	print ".AE"; print ".LP";
    }
    next;
}
    

/;/ {
    i=(i=index($0, ";") + index(substr($0, i), " "));
    split(substr($0, i-1), s, " "); # r=0, ul=1, bl=2
    printf("%s", substr($0, 0, i-2)); d=0;
    for(i in s) {
	f=""; g=""; w=s[i];
	switch(substr(w, 0, 1)) {
	case "*":
	    if(a[d] == 2) {
		if(--d == 0) f="\\fR";
		else if(a[d] == 1) f="\\fI";
	    }
	    else {
		f="\\fB"; a[++d] = 2;
	    }
	    w=substr(w, 2);
	    break;
	case "/":
	    if(a[d] == 1) {
		if(--d == 0) f="\\fR";
		else if(a[d] == 2) f="\\fB";
	    }
	    else {
		f="\\fI"; a[++d] = 1;
	    }
	    w=substr(w, 2);
	}
	switch(substr(w, length(w))) {
	case "*":
	    if(a[d] == 2) {
		if(--d == 0) g="\\fR";
		else if(a[d] = 1) g="\\fI";
	    }
	    else {
		g="\\fB"; a[++d] = 2;
	    }
	    w=substr(w, 0, length(w)-1);
	    break;
	case "/":
	    if(a[d] == 1) {
		if(--d == 0) g="\\fR";
		else if(a[d] == 2) g="\\fB";
	    }
	    else {
		g="\\fI"; a[++d] = 1;
	    }
	    w=substr(w, 0, length(w)-1);
	}
	printf(" %s%s%s", f, w, g);
    }
    print "";
    next;
}

/./ {print}

The structure looks a bit messy but the program works reliably.

I also don't see many people using ed in their scripts. This is an example, that changes cuesheet filenames and works with any number of files:
#!/bin/sh
file=$1; shift
printf '/^FILE/c\nFILE "%s" WAVE\n.\nw\n' "$@" | ed $file

 No.18498

does microsoft powershell count? its open source and cross platform with dotnet support as well

 No.18499

>>18498
The Powershell counterpart to 'sed s/a/b/g' is 'ForEach-Object { $_ -replace 'a', 'b' }'. This seems unneccessarily verbose.
It would be interesting to see if any anons use Powershell for something else than launch scripts. I remember writing a few .bat launchers when still using Windows.

 No.18500

awk is my shell superpower

 No.18503

>>18499
ive made powershell scripts at work to do shit like get data from apis/databases, manipulate text files, login to linux boxes and execute commands, and send emails.

 No.18504

>>18503
It looks like powershell has the same features as bourne shell with fewer special purpose commands and perl-esque syntax. While looking for redirection to stderr, i read that when it writes errors to stderr, they are ignored py powershell, because it has internal error handling with "objects".
How much of a difference does it make for powershell to pass around "objects" instead of raw bytes?

 No.18506

>>18504
Instead of thinking of it as a shell try thinking of it more like a dynamic language for the dotnet platform that can also do shell-like stuff. So you can have try/catch blocks, classes, inheritance etc. Powershell has access to the whole dotnet API and it's types are native to dotnet. So anything you could do in for example C# you can do in powershell (or almost anything).

 No.21547

File: 1694507893993.png (Spoiler Image, 5.79 MB, 3318x4263, 3196122_ratopombo_commissi….png)

The folllowing fetches a restricted newgrouds image by iterating over the ids in a closed interval. Currently it is most convenient to specify the range with the closest unrestricted submissions on an author's page (thumbnail names include the id).
#!/bin/sh
if [ $# -ne 3 ]; then echo Usage: ingidspn n m addr >&2; fi
n=`expr $1 + 1`; m=`expr $2 - 1`; a=`echo $3 | sed 's/.*newgrounds\.com\/art\/view\///'`
seq $n $m | {
while read i && test -z $c; do
 if wget https://art.ngfiles.com/images/`echo $i | sed 's/...$//'`000/${i}_`echo $a | cut -d/ -f 1`_`echo $a |cut -d/ -f 2`.png;
 then c=1; fi
done
}

Usage example:
ingidspn 3196120 3196123 https://www.newgrounds.com/art/view/ratopombo/commission-the-busty-dunmer-lady

As it is, this method works well for some images closely submitted to others by the author and takes a long time for most.
If newgrounds had a way to get an art submission by id (as the movie portal has), a script could instead use a binary search matching the closest two unrestricted images by submission time.
>inb4 just make an account
never
People may also find this useful for writing a gallery-dl style ripper.

 No.21548

>>18504
Adding to: >>18506 whom I agree with, it also let's you call stuff and get rich and easy to manipulate objects. It's pretty powerful. Powershell is really nice, I can't complain. Bash and other similar languages, like zsh and even fish fucking suck major balls, even just syntactically. They can't compare to powershell.

I guess you can use something like python as a shell and maybe it's not so bad, but then it's not the default system shell, so you'll have to use a bash-like language at some point.

All I want is typed shell commands. This "text streams" bullshit and ridiculous operators like
> if [ -r $1 ]
Bruh what the fuck is this? Don't you dare write
> if [1 ==$1]
Btw what even is
>if (( -r $1 ))
Ridiculous.

 No.21549

>>21548
'[' is an alias (not in the shell sense) for 'test'. Due to the design goals of 'test', simple conditional syntax can be more kludgy than in most programming languages.
Tom Duff's rc shell has a '~' matching command.
>if(~ $#* 0) {echo No arguments.}
IMO this is far more elegant. Test or find style dash operators are perfectly fine for more complex expressions though, like in your case where it's checking for file readability.

 No.21550

>>21549
I expect this to work:
if(someCondition) { someCommand; }
Or if you want to check for a files existence then something like:
if (doesFileExist("/path")){ …
Which in powershell you can kind of do but in bash you have the illegible syntax and unexpected behavior. I know there's historical reason for this, but I just want better. A proper programming language that isn't a pain to read, write, or maintain. If it's testable, even better.

 No.21553

>>21550
>if (doesFileExist("/path")){ …
Unix shells simply don't have algebraic function syntax. They also lack basic provisions for organizing internal functions, like namespaces, so 'test' kind of acts like a package containing a 'test -r' function.
If you can get over this, rc may be for you.
It has consistent whitespace processing, C-like statements, proper arrays/lists and no backticks. Plan9port or 9base runs on nearly every system.
http://doc.cat-v.org/plan_9/2nd_edition/papers/rc http://man.9front.org/1/rc

 No.21555

File: 1694531854763.png (208.2 KB, 1280x616, 1371499563013.png)

>>21550
if [[ -e "/path" ]]; then
…
fi

How is this illegible? Are you one of those imbeciles that thinks every language should look like C, because that's the only language they know?

 No.21556

>>21555
What's -e? Do I really have to read what the entire documentation of [[ (which btw is not the same as [ ).
Do you really know the difference between all these?
if [
if [[
if ((
if (
and also just if eg:
if command

Even if you do, which I doubt, it's ridiculous and hard to learn, hard to read, hard to maintain, and incredibly non-obvious.

 No.21557

>>21556
[ is the standard POSIX test, [[ is the bash keyword that has regex support and &&/||, (( is for arithmetic, and ( is the subshell. I know all of these because I have learned the language.

It's not hard to learn or to read and it would be obvious to you too, if you took the effort to learn it instead of just expecting it to be the same as that one language that you are familiar with.

 No.21558

I'm not claiming that Bash is perfect, it has tons of issues, but hating on this of all things is just bikeshedding, obviously you never learned the language and therefore can't say anything substantial about it.

 No.21559

>>21558
>>21557
Yes, I have not learned the language. I'm pointing to the fact that it's a hard language to maintain. In part due to it's foreign syntax and non-obvious parameters.

Point being:
> [[ -e
Is not
> [[ –file-exists
One is clearly more maintainable for someone who doesn't program in the same language all the time. Imagine if all languages had these obscure flags and pitfalls, which is one of the things that irks me to no end of PHP, for example, which also makes it a terrible language.

Is it isset() or is_null() or was it isNull() or was it null or the triple = null or empty() or was it isEmpty() or is_empty()?

Because I know a bit of PHP I know which exists and which one is "better" depending on the content of the parameters, but this is poor language design, which is part of the reason PHP sucks to write, read, and maintain.

I'm a software engineer. I write code on my day to day job in PHP, Go, Java/Kotlin, MySQL, PostgreSQL, bash, and a bunch of markup languages and configurations. That's the perspective I'm approaching this from. If you're a hobbyist that doesn't really need testable, and maintainable code then bash is obviously a powerful language that does it's job well.

Also, to bring back powershell, it really does feel like a modern scripting language where you can access basically any dot net library. You don't have to write intractable awk/sed commands to get your text to behave. You can do that with normal string libraries. Stuff like that.

Again, what I want is a powerful language that is easy to write, and maintain, where I can feel confident it will do what I expect it to do. Even better if I can introduce types to it.

 No.21560

>>21559
I regularly write bash at work. It's fine. You can significantly increase your confidence by actually learning the language.

 No.21561

>>21560
How do you test your scripts? What do you write bash for?

 No.21562

>>21561
It's internal tooling to setup, manage and interact with various emulators. It's not tested.

 No.21578

>>21561
nta but I have actually written small integration tests for some of the automation scripts i made that are auto-run on every new commit i make to the scripts.

they aren't really organized and i haven't made a proper test framework out of it or anything, but they basically set up a mock environment for dummy projects with a local git remote and ssh, run the automation, check expected responses with curl and filesystem tools etc then tear the test env down.

 No.21644

You can learn Bash at https://tldp.org/LDP/abs/html/index.html
You can learn Sed and AWK at https://www.grymoire.com/Unix/index.html

>>18498
The bad thing about PowerShell is that it has so long start up time (at least on M$ Windows), so it's not pleasant to use it as a login shell. It's also verbose language. Is there a good resource for learning PowerShell? I already know the very basics. The reason why I'm interested in PS is that it's something new instead of another version/extension of sh. I'm also interested in elvish, scsh (Scheme Shell) and other similar projects for this same reason.
* https://elv.sh
* https://github.com/scheme/scsh
* https://cliki.net/CLISP-Shell (bonus - Clisp as shell)
* https://wiki.archlinux.org/title/Shell#Alternative_shells (more?)

 No.21694

>>21644
>Is there a good resource for learning PowerShell
Xah Lee has a pretty good tutorial, I used to be a powershell skeptic before he made a convincing case for it
http://xahlee.info/powershell/index.html

As for other shells, rc also does a good job of addressing some of bash's infelicities. It has a cleaner syntax while hewing more firmly to the UNIX philosophy.

 No.21700

>>21644
Does anyone ever even used Powershell except for backdoor purposes? I never seen it.
Glad it's their though. MS really are retards.

 No.21709

>>21700
Powershell is heavily used in the IT industry to manage networked windows boxes.

 No.21710

>>21709
>>21700
you can also use it to provision AWS or Azure infrastructure along side python or terraform


Unique IPs: 12

[Return][Go to top] [Catalog] | [Home][Post a Reply]
Delete Post [ ]
[ home / rules / faq ] [ overboard / sfw / alt ] [ leftypol / siberia / hobby / tech / edu / games / anime / music / draw / AKM ] [ meta / roulette ] [ cytube / wiki / git ] [ GET / ref / marx / booru / zine ]