User Tools

Site Tools


import_process

This is an old revision of the document!


Import Process

This is generic overview of the import process from an HTML source into a database. I won't go into details or specifics because I don't want to support the code, plus it's hacked together anyway.

Here's pretty much how it operates though:

  • Fetch all HTML sources
  • Convert HTML sources to DOM
  • Insert a custom JavaScript source file using JQuery to remove HTML tags and get scripture text
  • Iterate over the modified HTML files
  • Custom JavaScript makes an AJAX call to a script which sanitizes data and imports it into the database

That's it!

Doing some code dumps here:

<script src='jquery-3.3.1.min.js'></script><script type='text/javascript'>var v=1;var scripture=new Object();for(v=1;v<$('p.verse').length+1;v+=1){$('#p'+v+' span.verse-number.verse').remove();$('#p'+v+' span.para-mark').remove();$('#p'+v+' sup.studyNoteMarker.dontHighlight').remove();scripture.bcv=$('title').text()+':'+v;scripture.text=$('#p'+v).text();$.ajax({url:'http://localhost:8080/',type:'POST',async:false,data:scripture})}</script>
import_process.1553465111.txt.gz · Last modified: 2019/03/24 16:05 by steve