Browser Automation and Web Scraping – References

These are references from my portfolio — solutions built for specific clients — in which a program does what a person would do in a browser: it logs in to websites, walks through their steps, fills in forms, sends messages and collects data from pages. It does not matter whether the data comes from an Excel sheet, a Word document or a business system, or where it ends up; what they share is that they work with someone else's web applications that have no API and can change without notice. I do this kind of work as part of my Windows programs and background services development service, and when the solution relies on Office, as part of Office add-in development.

A robot that fills in web forms, driven from a business system

Built in the fourth quarter of 2025.

What was built

A Windows program that runs in the background and drives the browser instead of an operator: it opens the assigned websites, walks through their steps and fills their forms with data that already exists in the TsatsaTayka business system. The outcome of every pass is written back, so the records always show how far the work has got.

What sets it apart is not that it can click around a page — every automation tool can do that — but where its procedure is kept. The steps are not written into the program; they are held as data in the business system, in ordinary tables edited in the browser.

Why the procedure is data and not code

Somebody else's website changes without notice: a field moves, a step is added, the text on a button changes. With a conventional robot every such detail means changing the program, releasing a new version and installing it again at the client.

Here it means changing one row in a table. Even the way a field is located on the page is held in that table, together with the value written into it, so a changed website is handled by correcting data. The program stays the same. The same applies to adding a new website or a new kind of job — a new procedure is added, not a new program.

Every step also carries the condition under which it runs, so one procedure covers branching cases without a separate variant for each. A step can also carry the program version it applies from, so changes are prepared in advance and do not disturb installations that have not been updated yet.

What the client gets

  • Work that happens without a person – tasks are processed in cycles, on their own, including overnight.
  • Changes without a developer – when a website changes, an authorized user corrects the step in the table and work continues.
  • One system instead of three – intake, records and processing are connected, and nothing is retyped.
  • A trace of every pass – screenshots, a work log, and a record of what was changed and when.
  • Several programs on several machines – each installation takes only the kind of work assigned to it, so the load is shared.

When something goes wrong

Automation that works against somebody else's website occasionally meets something unforeseen. The program was therefore not built to simply stop when that happens.

The operator has controls to start, pause and stop, can be asked to supply manually what is missing and then continue from the same point, and the behaviour on an error is a choice: stop, pause for inspection, or continue with an entry in the log. A screenshot is kept at every questionable point, so it can be seen afterwards what the website actually displayed.

Where the limit is

The program does not decide on behalf of a person and does not bypass the protections of the websites it works with — it does what the operator would do anyway, only without retyping and without waiting. Checks that need human judgement stay with the operator, and the program brings them up rather than skipping them.

Where this kind of solution is used

Anywhere the same external form is filled in daily with data that already exists in some record: applications and requests on portals with no API of their own, lookups in registers, status checks, or moving orders from one system to another.

That makes this the clearest example of what TsatsaTayka is for: the system holds the data, the rules and the records, and the installed program is only the hand that carries them out.

Which technologies it exists in

In use is the variant written in VB.NET, which is the one delivered to the client.

A C# variant has been worked out as well. Because the whole procedure sits in the tables of the business system, both variants run on the same data — the one chosen is whichever fits better with what the client already uses and maintains.

Planned is a version as an Electron application in JavaScript, doing the same work outside Windows. The split between the system that holds the procedure and the program that carries it out is exactly what makes that possible: the executor changes, not the rules it works by.

Technologies used

The program is written in VB.NET and C# with a WinForms interface, drives Microsoft Edge through Selenium WebDriver, and talks to the business system over a REST API. The business system is PHP and MySQL, with Tabulator.js tables for editing the procedures. It ships with a Windows installer, and an Electron version in TypeScript is in preparation.


Sending SMS messages by driving someone else's web application

Built in the third quarter of 2024.

A desktop application driving a browser in place of the user

This is a reference — a solution built for one client and described the way it came about. If you need a ready-made application for sending SMS messages from Excel, it exists as a separate product: send SMS from Excel.

The task

The client had a job that came back every month: several hundred messages from a table, each to be sent separately, with a record of what went out. A service that sends SMS over the internet was not an option — the messages had to leave from their number, from their phone, and through their own mobile plan.

That leaves one road only: an application that does what a person would do by hand, in a browser, but without the person.

Why that is harder than it looks

A program driving someone else's web application has no contract with it. There is no API guaranteeing that tomorrow works like today, no error message it can rely on, no promise that the text field and the send button will stay where they are. All such a program has is the screen — the same one the user sees.

Three problems follow from that, and they are the work:

  • Waiting. A web application does not announce that it is ready. A program starting too early types into a field that does not exist yet, and a program waiting a fixed five seconds per message spends half an hour on a list that takes ten minutes.
  • Breaking off halfway. If the connection drops on the hundred and fiftieth message, the job must neither be abandoned nor started over. So the record of what was sent does not live in the program's memory but in the table itself, row by row, at the moment of sending — which means a restart continues where it stopped.
  • Pairing the device. The web version of the messages works only while the phone is paired with the computer, and that pairing needs a person. So that one step was deliberately left to the user, and everything else runs without them.

How it was solved

A Windows desktop application that walks the browser through the steps the user would take by hand, using the table both as its input and as its record. Everything that can change in someone else's web application — field names, buttons, waiting times — was moved out of the program and into settings, so a change on their side means editing one setting rather than shipping a new version.

Before the first send the application checks whether the environment is ready and, if it is not, says what is missing — because an error reported before the start is worth far more than an error in the hundred and fiftieth row.

Technologies used

  • C# programming language (programiranje.co.rs): the Windows desktop application.
  • Browser automation: walking the web version of the messages through the steps a user would take by hand.
  • Excel sheet: the message input and the record of what was sent.

What the client got

A job that used to take a working afternoon now takes as long as the sending itself, and the record of what went out and when is created along the way instead of being kept afterwards. The solution has since been polished into a finished product, sold under send SMS from Excel.


Pulling data from a web page into a Word document

Built in the fourth quarter of 2021.

Problem

The client needed a VB script embedded in a Word document that fetches various pieces of data from a given web page and writes them into specific fields in the document. Which part of the page is taken was to be set by the start and end HTML tags that the data sits between. The job was conceived as relatively simple, but the requirement that the script also work as an add-in increased its scope and complexity.

Solution

A script was built that pulls content from the web and fills in the Word document on its own:

  • dynamic data extraction: the script fetches any text placed between the given start and end HTML tags;
  • URL-based extraction: the process starts by entering the web page address;
  • writing directly into Word: the fetched data is placed into the designated fields of the document;
  • an add-in variant: the script was extended so it also works as an add-in in the Word environment.

Technologies used

  • VB script: automating tasks within the Microsoft Office suite.
  • HTML: the tags that define what is extracted from the page.
  • Microsoft Word: the document the data is written into.

Result

Data from the web page reaches the Word document on its own, without copying, and by changing the tags the same script can fetch other parts of the page.


Edge browser robots driven from Excel

Built in the third quarter of 2021.

Problem

The client already had a robotic automation system in the browser, which needed significant improvements to the first robot and a completely new second robot. The automation was to be controlled directly from an Excel add-in, with the working speed and the pauses between tasks adjustable while running. The biggest challenge was reliably extracting data from Excel and passing it through the forms of an application-submission website. Since the client's database was not available for testing, the robots had to be verified without access to the real system.

Solution

  • improved first robot: refined according to the given instructions, with additional capabilities;
  • a new second robot: developed with its own functions, so it works together with the first;
  • control from Excel: data from the sheet is extracted automatically and used to fill in the forms on the website;
  • adjustments while running: a gear icon changes execution speed and the pauses between tasks in real time;
  • a test environment: a temporary page replicating the real platform was built so the robots could be tested faithfully.

Technologies used

  • VB.NET programming language: the basis of the robotic automation.
  • Selenium: controlling the Edge browser.
  • Microsoft Excel: the main data source for the robots.

Result

The client controls two robots from Excel, adjusts their pace without stopping the work, and both were verified in an environment that faithfully follows the real website.


A VSTO add-in that automates work on websites

Built in the third quarter of 2021.

Problem

The client had a series of web steps that needed to be automated: logging in to a website, processing HTML tables, clicking buttons, downloading Excel files, moving to another website, and filling in and submitting forms. They asked for a VSTO add-in in VB.NET to do that work for them.

Solution

An add-in was built that simplifies web-based processes and reduces manual work:

  • automating work on websites: logging in, processing table data, downloading files and filling in forms;
  • working with page elements: entering data and submitting forms on the target websites;
  • an add-in in the Office environment, so the work is started from an application the user already uses.

Technologies used

  • VSTO (Visual Studio Tools for Office): the framework for building the add-in in the Microsoft Office environment.
  • VB.NET: the language the add-in is written in.
  • Selenium: working with page elements on the target websites.

Result

A series of steps across several websites, from login to form submission, is started from Office and runs without manual clicking.


A two-level page data scraper

Built in the third quarter of 2021.

Problem

The client needed a scraper to extract company data from a search engine on one website. The scraper had to go through the site's search, enter the parameters set by the user — departure location, destination location and city — and collect the list of results, then open each company's page and fetch contact details and other information.

Solution

A precise and efficient scraper was built that collects data from two levels of pages:

  • two levels of pages: data is extracted both from the search results page and from the individual detail pages;
  • search parameters: the user enters the locations and city and gets the matching list;
  • detailed data: beyond the list, the scraper opens every link and fetches contact and other details;
  • export: all data is organized into clear Excel and CSV files, ready for analysis.

Technologies used

  • Web scraping tools: accurate extraction of data from pages.
  • Excel and CSV: the formats the data is exported to.

Result

For a given search, the client gets a complete table of companies with contact details without opening pages by hand, and the work was finished on time.

Client feedback

★★★★★5/5

“Dejan was a star. He completed all the assigned tasks on time and was very patient and thorough with his communication and documentation. Thank you, Dejan!”

Do you need similar automation?

What development includes, how the price is set and how the work proceeds is described on the Windows programs and background services page, and for connecting to other systems that do have an API, on the API integrations page. See the ready-made messaging application on the send SMS from Excel page, and find other references among the Windows automation software examples.