Not being able to query a new corpus.

CorText Manager Q&A forumCategory: Data explorationNot being able to query a new corpus.
matias.milia asked 6 years ago

Hi, I have tried all but I can’t find a way to extract a subcorpus and build a new database based on the custom time steps I have build with period detector. I am trying to do this in the last time step (number 2; 2008-2016) which is the most larger one and represents around the 75% of the whole database. According to my calculations, this would add up to around 200k entries. This are all the strategies I’ve tried:
Applied to ISIpubdate_custom:
data = ‘2’
data = ‘%2%’
data like ‘%2%’
Same field but with opposite condition:
data IN (‘0’, ‘1’)
data = ‘0’ OR ‘1’
Applied to the ISIpubdate:
data = ‘2008’ OR ‘2009’ OR ‘2010’ OR ‘2011’ OR ‘2012’ OR ‘2013’ OR ‘2014’ OR ‘2015’ OR ‘2016’
I manage to get one database but it was empty. Also, I manage to get databases for the first two periods 0 and 1; they work just fine. So I am stuck here, don’t know how to go around this. I always get this message:

Debug Log:
Error! Log file not found.
Start your code here

Any idea on how to get around this problem?

matias.milia replied 6 years ago

Just corrected the boolean terms applied to ‘ISIpubdate’, but still won’t make it. Checked on Corpus Explorer, and the data is there; so, no clue of what is going on. I copied year numbers and period numbers to avoid ‘hidden characters’ but it won’t work either.
Here the full script log, just in case.

2018-11-17 16:44:01 INFO : Script Query Corpus Started
2018-11-17 16:44:01 INFO :
Query Parameters:
Query Type: sql
target table name: ISIpubdate
condition: data = ‘2008’ OR data = ‘2009’ OR data = ‘2010’ OR data = ‘2011’ OR data
= ‘2012’ OR data = ‘2013’ OR data = ‘2014’ OR data = ‘2015’ OR data = ‘2016
Inverse the condition: false
Build an entirely new database excluding articles that do not match the query for every fields (*): true
output table/bdd name: Time2ok

2018-11-17 16:46:08 INFO : Executed query:INSERT INTO ISIAUTHORselid ( file, id, CompteDedata ) SELECT ISIpubdate.file, ISIpubdate.id, Count(ISIpubdate.data) AS CompteDedata FROM ISIpubdate WHERE ISIpubdate.data = ‘2008’ OR ISIpubdate.data = ‘2009’ OR ISIpubdate.data = ‘2010’ OR ISIpubdate.data = ‘2011’ OR ISIpubdate.data = ‘2012’ OR ISIpubdate.data = ‘2013’ OR ISIpubdate.data = ‘2014’ OR ISIpubdate.data = ‘2015’ OR ISIpubdate.data = ‘2016’ GROUP BY ISIpubdate.file, ISIpubdate.id
2018-11-17 16:46:15 INFO : Creating a new database

matias.milia replied 6 years ago

Period slicer says, that has ‘ correctly finished ‘ .
I manage so see with ‘corpus explorer’ that at the ‘Pubdate_Cu’ (Custom Pub Date) field everything is ok. The value is correctly set at ‘2’.

—-
2018-11-05 22:13:13 INFO : Period Slicer Started
2018-11-05 22:13:13 INFO :
Period slices definition:
Enter a custom time partition: ‘[1991:2000];[2001:2007];[2008:2016]’

2018-11-05 22:13:15 INFO : Creating time slices according to this time division: [‘1991_2000’, ‘2001_2007’, ‘2008_2016’]
2018-11-05 22:13:17 INFO : New Time table ISIpubdate_custom created
2018-11-05 22:13:17 INFO : Period Slicer correctly finished

matias.milia replied 6 years ago

Actually, just managed to access to cluster composition in the ‘network analysis’ based on specific time periods (detected with period slicer). Everything is at ‘maps_output.zip’ at the folder ‘clusters’. I don’t know why I didn’t see this before, I know I looked for it; but sometimes it happens.
I will keep on working on this data, so the period slicing is not so relevant for the moment. Nevertheless, I have been experimenting problems with the script and this database before. Since I will probably still get them in the future, I leave this question open.

matias.milia replied 6 years ago

Actually, just managed to access to cluster composition in the ‘network analysis’ based on specific time periods (detected with period slicer). Everything is at ‘maps_output.zip’ at the folder ‘clusters’. This was the main goal of trying to build new databases. I don’t know why I didn’t see this before, I know I looked for it; but sometimes it happens.
I will keep on working on this data, so the period slicing is not so relevant for the moment. Nevertheless, I have been experimenting problems with the script and this database before (when trying to slice using the ‘country’ field). I got support on this, should use for example:  data like ‘%Mexico%’. Last time I tried I got some sort of error as well. I will try it again in order to see if the error persist or not. If it doesn’t, I will close this question.