World: r4wp
[Databases] group to discuss various database issues and drivers
older newer | first last |
TomBon 16-Nov-2012 [350] | nice, good luck with your crossword afsan... |
BrianH 16-Nov-2012 [351] | With parameterized queries (even in REBOL) the SQL and the parameters are sent separately and combined in the server. The query plan is generated only once per query, with the parameter placeholders being in the plan. Then the actual parameters are plugged into the plan. The next time the parameterized query is called (maybe with differe3nt parameter values) the same plan is used and the new parameter values are plugged in. |
TomBon 16-Nov-2012 [352] | isn't this execution optimation?. in this case a stored procedure will help also. how will this prevent from sql injection? compared to a concatenated server side sql string? |
BrianH 16-Nov-2012 [353x2] | If you build a query dynamically with rejoin or something, the query is put together client side and then the server has to generate a new query plan for each distinct set of parameter values. This takes time and blows the query plan cache, which slows down the whole query process. |
The problem is that your ad-hoc parameter screening is usually not perfect. Parameterized queries don't build a query in the server, they just plug in the values to an already-compiled query (the "query plan"). The server doesn't have to do any parameter screening other than for malformed values in the protocol. | |
TomBon 16-Nov-2012 [355] | depends on the needs. I always try to detach the data sink from input logic. this way you can change your db backend very easy but of course everybody has it's own style in this. |
BrianH 16-Nov-2012 [356x2] | For new developers ad-hoc parameter screening is even more likely to be bad (and most that don't use parameterized queries are still new, no matter how long they've been programming, because parameterized queries are almost always inherently better). Even if it wasn't a safety issue, they're a lot faster. |
I've seen data front-ends that don't use parametrized queries when talking to SQL servers that support them. They need work. | |
afsanehsamim 16-Nov-2012 [358] | could you tell me how can i compare values of two tables in database? |
Endo 17-Nov-2012 [359] | About parametrized queries: The only problem using them on R2, at least with RT's default ODBC, there is no chance to use NULL values. None of those work: insert db-port ["INSERT t (a) VALUES (?)" NULL] insert db-port ["INSERT t (a) VALUES (?)" 'NULL] insert db-port ["INSERT t (a) VALUES (?)" "NULL"] insert db-port ["INSERT t (a) VALUES (?)" none] insert db-port reduce ["INSERT t (a) VALUES (?)" none] |
TomBon 17-Nov-2012 [360] | you have more than one solution, the first is a simple serial SELECT on each table -> compare the output for equal. of course this produce unnecessary DB overhead but I guess you won't feel any speed difference except you are serving a whole city concurrently. another, better one is a JOIN or UNION. SELECT table_name1.column_name(s), ... FROM table_name1 LEFT JOIN table_name2 ON table_name1.column_name=table_name2.column_name the JOIN direction (LEFT,RIGHT,INNER) for your reference table is important here. the resultset is a table containing BOTH columns. if both having a value -> match, if one is empty then you don't. index both fields to accelerate the query and use something like the free SQLyog to test different queries to make debugging easier for you. while you situation reminds me to myself, sitting infront of a monochrom asthon tate dot some decades ago and asking what next?, you should 'bite' yourself now thru the rest. It won't help you on longterm if you don't. |
afsanehsamim 17-Nov-2012 [361x11] | @TomBon: my query for joining two tables is :insert db["select * from data LEFT JOIN data1 ON data.oneone=data1.oneone"] and output is :[ ["c" "a" "t" "a" "e" "r" "o" "a" none none none none none none none none] ] plz tell me what should i write in query that i get values instead of none in output ? |
guys when i enter correct value in form the above join query works properly... i need help for writing queries which other condition,it means if user enter wrong value ,it joins with first table but dose comparing indicidually and shows error message. | |
the output of this query insert db[{select * from data LEFT JOIN data1 ON data.oneone=data1.oneone}] is : [ ["c" "a" "t" "a" "e" "r" "o" "a" "c" "a" "t" "a" "e" "r" "o" "a"] ] | |
is there anyone who can help me ?? | |
i compare each field of tables with each other like :insert db["select data.oneone,data1.oneone from data LEFT JOIN data1 ON data.oneone=data1.oneone"] results: copy db probe results insert db["select data.onetwo,data1.onetwo from data LEFT JOIN data1 ON data.onetwo=data1.onetwo"] results: copy db probe results insert db["select data.onethree,data1.onethree from data LEFT JOIN data1 ON data.onethree=data1.onethree"] results: copy db probe results ..... | |
i got result | |
i need codes for showing message to user ,it mean after each joining ,it should show user that value is correct or no | |
guys ! could you plz tell me after comparing values of two tables how can we show the output on web page? | |
after writing queries :foreach row read/custom mysql://[root-:-localhost]/test ["select data.oneone,data1.oneone from data LEFT JOIN data1 ON data.oneone=data1.oneone"] [print row] foreach row read/custom mysql://[root-:-localhost]/test ["select data.onetwo,data1.onetwo from data LEFT JOIN data1 ON data.onetwo=data1.onetwo"] [print row] .... | |
i got this results:c c a none t t a none e none r none o none a none | |
now how can i write query for everyvalues which are same and print correct message on web page? | |
afsanehsamim 23-Nov-2012 [372x2] | hey guys... i have just 2days time for my project ! could you help me? |
i could not do the last step ... i should show result of comparing values on web page | |
TomBon 11-Dec-2012 [374] | a quick update on elasticsearch. Currently I have reached 2TB datasize (~85M documents) on a single node. Queries now starting to slow down but the system is very stable even under heavy load. While queries in average took between 50-250ms against a dataset around 1TB the same queries are now in a range between 900-1500 ms. The average allocated java heap is around 9GB which is nearly 100% of the max heap size by a 15 shards and 0 replicas setting. elasticsearch looks like a very good candidate for handling big data with a need for 'near realtime' analysis. Classical RDBMS like mysql and postgresql where grilled at around 150-500GB. Another tested candidate was MongoDB which was great too but since it stores all metadata and fields uncompressed the waste of diskspace was ridiculous high. Furthermore query execution times differs unexpectable without any known reason by factor 3. Tokyo Cabinet started fine but around 1TB I have noticed file integrity problems which leads into endless restoring/repairing procedures. Adding sharding logic by coding an additional layer wasn't very motivating but could solve this issue. Within the next six months the datasize should reached the 100TB mark. Would be interesting to see how elasticsearch will scale and how many nodes are nessesary to handle this efficiently. |
Maxim 11-Dec-2012 [375] | when you talk about "documents" what type of documents are they? |
Gregg 11-Dec-2012 [376] | Thanks for the info Tomas. |
TomBon 12-Dec-2012 [377] | crawled html/mime embedded documents/images etc. as plain compressed source (avg. 25kb) and 14 searchable metafields (ngram) to train different NN types for pattern recognition. |
Maxim 12-Dec-2012 [378] | thanks :-) |
MaxV 15-Jan-2013 [379] | I have a problem with RebDB: how works db-select/group? Example: >> db-select/where/group/count [ID title post date] archive [find post "t" ] [ID] ** User Error: Invalid number of group by columns ** Near: to error! :value |
Endo 15-Jan-2013 [380x2] | Don't you need to use aggregate functions when you grouping? |
* when you use grouping. | |
Scot 15-Jan-2013 [382x3] | I use the sql dialect like this: sql [select count [ID title post date] from archive group by [ID title post] where [find post "t"]] The trick with this particular query is the that the "count" selector must have exactly one more column than the "group by" selector. The first three elements [ID title post] are used to sort the output and the last element [date] is counted. output will be organized: ID title post count |
I would like to be able to include other columns in the output that are not part of the grouping or count, but I haven't figured out how to do this in RebDB. I have used a parse grammar on the output to achieve the desired result. | |
I would also like to query the results of a query, which I haven't figured out how to do so without creating and committing a new database. So I have used a parse grammar to merge two queries. | |
Pavel 25-Jun-2013 [385] | SQLite version 4 announced/proposed. The default built-in storage engine is a log-structured merge database instead of B-tree in SQlite3. As far as I understand the docs This store could be usable standalone or use SQL frontend. Google to SQLite4. |
Kaj 25-Jun-2013 [386] | Cool |
Endo 26-Jun-2013 [387] | I cannot see any announcement on the sqlite.org web site? SQLite 3.7.17 is the latest and recommended version? |
Kaj 26-Jun-2013 [388] | I saw code last year, but it's probably still in deep development |
Pavel 26-Jun-2013 [389] | Endo as I wrote google for SQLite4. direct link is: http://sqlite.org/src4/doc/trunk/www/design.wiki. There is a mirror of souces at https://github.com/jarredholman/sqlite4 also. |
Pekr 4-Jul-2013 [390x4] | Has anyone tried to work with ODBC under R3? I somehow can't load following ODBC driver DLL: https://github.com/gurzgri/r3-odbc |
Or differently, has anyone worked with excel files via ODBC, using either R2 or R3? I tried Graham's code, which works for .xls files, but not .xlsx files. When I convert my file to .xls, R2 returns - not enough memory :-( p: open [ scheme: 'ODBC target: "Driver={Microsoft Excel Driver (*.xls)};DriverId=790;Dbq=c:\path-to-file\file.xls" ] conn: first p insert conn "select * from [Sheet1$]" result: copy conn | |
As for R3 - maybe there was also some other R3 ODBC extension, somehow can't find it .... | |
hmm, found it, but no more available - http://www.rebol.org/ml-display-thread.r?m=rmlXJPF ... the problem with gurzgri DLL is, I can't somehow import it with any R3 version ... | |
Kaj 4-Jul-2013 [394x3] | What you found looks to be the latest version of that |
I've also had loading problems with R3 extensions on Linux that worked before. Sometimes you seem to need an older R3, sometimes a newer | |
If all else fails, recompile the C code | |
Pekr 4-Jul-2013 [397] | well, I have even old latest Carl's view.exe, does not work either ... lost battle here ... not fluent with recompile of ODBC DLL, does not imo guarantee, that loading it in R3 will actually work. I wonder if there was any change to import function or to extension mechanism itself ... |
Kaj 4-Jul-2013 [398] | Bug fixes, I think, but they also seem to cause compatibility regressions |
DocKimbel 4-Jul-2013 [399] | Do all your binding have Red-level interfaces now? |
older newer | first last |