4.1.2.4. Scraping bot manuel

Not all sources can be inserted using the “point & click” method described in the previous tutorial.

This can be prevented:

• because the display does not load
• clicking on an item is not possible
• the robot doesn’t understand which specific box you’re interested in.
  

However, it is often possible to +manually edit these selectors.

*If you don’t have a basic knowledge of HTML, mastering this aspect requires a certain amount of practice and time. The Cikisi team offers training modules to help you master scrapping bot in expert mode, enabling you to set up sources on more than 90% of existing web pages.

To do this, you need to know some basic HTML.
HTML is the language of web pages in the form of tags.

How do developers design a web page?

They create boxes, nest them inside each other and then inject content (URL links, images, text, etc.) into them.
The aim of the game is to find which box contains the contents of a selector and to indicate this to the Cikisi robot.

These boxes are of a certain type, called “tag html”. You’ll use a dozen of them at most, and they’re often the same for the same selectors.

Here are the most common:

• h1, h2, h3, h4, h5 and h6: h for “header”. These are boxes for headings. “h1” will be the largest possible heading and h6 the smallest.
• p: for “paragraph”, contains text.
• img: a box for images (95% of “image” selectors).
• a: a box for URL links. 100% of “link” selectors.
• div: a catch-all box. When a developer wants to store content somewhere, but has no idea where to put it (because there’s no dedicated box type), he stores it in a “div”.
• ul: a bulleted list of several items (a recipe, for example)
• li: inside an “ul”, the points of the list.
  

How do I find the type of box I’m interested in?

• Open the page you’re interested in.
• Right-click* on the element of your choice (a title, for example).
• Click on “inspect” at the very bottom.
• The “boxes” are marked in pink in the pane that opens on the right.