Mass search-and-replace blocks of text (hundreds of files)

If it's not ZDoom, it goes here.
User avatar
Nash
 
 
Posts: 17296
Joined: Mon Oct 27, 2003 12:07 am
Twitch ID: nashmuhandes
Github ID: nashmuhandes
Location: Kuala Lumpur, Malaysia

Mass search-and-replace blocks of text (hundreds of files)

Post by Nash »



I'm looking for a way to easily do search-and-replace for huge text blocks like in the image above... across hundreds of files.

All text editors I have can only do simple 1-line search-and-replace.
dpJudas
 
 
Posts: 2869
Joined: Sat May 28, 2016 1:01 pm

Re: Mass search-and-replace blocks of text (hundreds of file

Post by dpJudas »

I think Visual Studio's Replace in files can do this, but it gets a bit annoying because you will have to use the regular expression feature. Something like searching for:

// Copyright \d+ Nash Muhandes.*\n(//[^\n]*\n)*

Now how to then replace it with a new multi-line thing I'm not sure how to do. Maybe the replace thing also supports \n when it regular expression mode? I think most people do these kinds of replaces with shell scripts or write a custom app to do it. Personally I would do it as a custom C# app as that would be faster for me to figure out the syntax than use a shell script with unholy combination of grep, awk, bash, etc.
User avatar
MartinHowe
Posts: 1962
Joined: Mon Aug 11, 2003 1:50 pm
Location: Waveney, United Kingdom

Re: Mass search-and-replace blocks of text (hundreds of file

Post by MartinHowe »

If there is no GUI tool to do this, it probably needs bash and sed and a Linux box/VM or Win32 implementations of same in PS or CMD.

1 Put each block of text and its replacement in a file by itself
2 Run sed over them to convert every CR to '\r' and every LF to '\n' (or '%0D' and '%0A')
3 Run sed over a copy of each file to be processed, to do the same
4 Create a script that uses sed with each text block as a very long string to be replaced and hope you don't run up against line length limits
5 Run the script on each file.
6 Reverse the process on the output files

This is the kind of stuff I have done in the past; there's probably better ways to do it, or optimisations you can do depending on which version of sed you have access to.

I'm too overloaded with my own stuff to spend time developing and debugging this, but hopefully the above is a possible direction.

Post-ninja EDIT:
dpJudas wrote:than use a shell script with unholy combination of grep, awk, bash, etc.
Oh I love making abominations of bash, sed, awk, etc. Just hope DoomGuy doesn't try to kill them :p
dpJudas
 
 
Posts: 2869
Joined: Sat May 28, 2016 1:01 pm

Re: Mass search-and-replace blocks of text (hundreds of file

Post by dpJudas »

Oh and since I was waiting for a coworker, to illustrate just why I would opt for the C# solution over a shell script. Here's the code:

Code: Select all

string oldText = @"
// Copyright 2021 Nash Muhandes
// Blah blah blah
";

string newText = @"
// Copyright 2022 The Easter Bunny
// More blah!
";

void scanFolder(string path)
{
    foreach (string filename in Directory.GetFiles(path))
    {
        string text = File.ReadAllText(filename);
        text = text.Replace(oldText, newText);
        File.WriteAllText(filename, text, new System.Text.UTF8Encoding(false));
    }

    foreach (string folder in Directory.GetDirectories(path))
    {
        scanFolder(folder);
    }
}

scanFolder("C:\\Development\\NashStuff");
Much easier than using old 1970's tech if you ask me. :)
User avatar
Tormentor667
Posts: 13461
Joined: Wed Jul 16, 2003 3:52 am

Re: Mass search-and-replace blocks of text (hundreds of file

Post by Tormentor667 »

I am pretty sure that Notepad++ is capable of doing that.
User avatar
Kizoky
Posts: 284
Joined: Mon Nov 14, 2011 9:59 am
Discord: Kizoky#0375
Github ID: Kizoky
Graphics Processor: nVidia with Vulkan support
Location: Around weirdos, I'm the biggest weirdo among them

Re: Mass search-and-replace blocks of text (hundreds of file

Post by Kizoky »

Tormentor667 wrote:I am pretty sure that Notepad++ is capable of doing that.
It doesn't support multi-line replacing AFAIK, but it is indeed capable of looking for matched cases and replace them
User avatar
Nash
 
 
Posts: 17296
Joined: Mon Oct 27, 2003 12:07 am
Twitch ID: nashmuhandes
Github ID: nashmuhandes
Location: Kuala Lumpur, Malaysia

Re: Mass search-and-replace blocks of text (hundreds of file

Post by Nash »

Yeah no, Notepad++ won't do. I need to replace, in 1 click, a massive block of text (multiple lines), over hundreds of files.

dpJudas' C# app will probably do the job for me. Could probably change it to read a text file for the old and new, so that I don't have to keep recompiling it all the time. Thanks!
Edward-san
Posts: 1774
Joined: Sat Oct 17, 2009 9:40 am

Re: Mass search-and-replace blocks of text (hundreds of file

Post by Edward-san »

Hey there!

If you do not care about the end line style, with Visual Studio Code it seems I can do that without any special character handling, just copy-paste the text into the 'Find' box. See the image below for an example.

User avatar
Nash
 
 
Posts: 17296
Joined: Mon Oct 27, 2003 12:07 am
Twitch ID: nashmuhandes
Github ID: nashmuhandes
Location: Kuala Lumpur, Malaysia

Re: Mass search-and-replace blocks of text (hundreds of file

Post by Nash »

I indeed do use VS Code as my main IDE for ZScript and holy crap, I didn't know I could do that!
User avatar
Xeotroid
Posts: 385
Joined: Sat Jun 23, 2012 7:44 am
Graphics Processor: nVidia with Vulkan support
Location: Czech Rep.

Re: Mass search-and-replace blocks of text (hundreds of file

Post by Xeotroid »

It's definitely quite easily possible to replace multiple lines at once in multiple files with Notepad++. I don't know of an advantage of doing so instead of using VSCode (no experience there), but it's the first option I would go around trying. See here: https://files.catbox.moe/xbgu5m.webm
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 48034
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: Mass search-and-replace blocks of text (hundreds of file

Post by Graf Zahl »

Yeah, right, you have to screw up your text file first. I'd rather use an editor capable of doing it natively...
User avatar
Xeotroid
Posts: 385
Joined: Sat Jun 23, 2012 7:44 am
Graphics Processor: nVidia with Vulkan support
Location: Czech Rep.

Re: Mass search-and-replace blocks of text (hundreds of file

Post by Xeotroid »

Looks like selecting the text and going to Find+Replace automatically copies the whole selection including line breaks into the Find field*, so the preprocessing is not necessary. Copy-pasting the text manually there is not possible, however – for whatever reason, only the first line gets pasted. You still need to use escape sequences in the Replace field too. It works well enough, but it's not ideal.

*Which is usually more annoying than useful for me, because I select blocks of text to limit where F+R takes place, not what I want to find.

Return to “Off-Topic”