Showing posts with label hiwe. Show all posts
Showing posts with label hiwe. Show all posts

Monday, March 19, 2012

Limits on rows in a table

Hi

We received the following in an email from a third-party supplier (who
naturally has a solution for the problem as described). It sounds like
gibberish to me, but does anyone have any comments?

<quote>
SQL in its current incarnation hits a performance brick wall when a
table contains more than about 75 million rows. This is not a
configuration limit as the table could be grown a lot larger but the
performance issue generates problems for ??; primarily during
search and retrieval of archived objects; although if the database
engine is being heavily hit for retrieval the archiving process can
slow down as well.
</quote
Chloe Crowder
British Librarychloe.crowder@.bl.uk wrote:
> Hi
> We received the following in an email from a third-party supplier (who
> naturally has a solution for the problem as described). It sounds like
> gibberish to me, but does anyone have any comments?
> <quote>
> SQL in its current incarnation hits a performance brick wall when a
> table contains more than about 75 million rows. This is not a
> configuration limit as the table could be grown a lot larger but the
> performance issue generates problems for ??; primarily during
> search and retrieval of archived objects; although if the database
> engine is being heavily hit for retrieval the archiving process can
> slow down as well.
> </quote
I've never heard something like this before. Maybe it's a problem
limited to a specific application. Maybe the app uses the DB in bad
ways (wrong indexing...)?

robert|||chloe.crowder@.bl.uk wrote:
> Hi
> We received the following in an email from a third-party supplier (who
> naturally has a solution for the problem as described). It sounds like
> gibberish to me, but does anyone have any comments?
> <quote>
> SQL in its current incarnation hits a performance brick wall when a
> table contains more than about 75 million rows. This is not a
> configuration limit as the table could be grown a lot larger but the
> performance issue generates problems for ??; primarily during
> search and retrieval of archived objects; although if the database
> engine is being heavily hit for retrieval the archiving process can
> slow down as well.
> </quote>
> Chloe Crowder
> British Library

"SQL in its current incarnation..."

SQL is a language and it has no such limitations. Did they mean to
refer to Microsoft SQL Server? Either way, their claim is plain
nonsense.

--
David Portas, SQL Server MVP

Whenever possible please post enough code to reproduce your problem.
Including CREATE TABLE and INSERT statements usually helps.
State what version of SQL Server you are using and specify the content
of any error messages.

SQL Server Books Online:
http://msdn2.microsoft.com/library/...US,SQL.90).aspx
--|||(chloe.crowder@.bl.uk) writes:
> We received the following in an email from a third-party supplier (who
> naturally has a solution for the problem as described). It sounds like
> gibberish to me, but does anyone have any comments?
><quote>
> SQL in its current incarnation hits a performance brick wall when a
> table contains more than about 75 million rows. This is not a
> configuration limit as the table could be grown a lot larger but the
> performance issue generates problems for ??; primarily during
> search and retrieval of archived objects; although if the database
> engine is being heavily hit for retrieval the archiving process can
> slow down as well.
></quote
SQL [Server] in its current incarnation? Nah, rather the current
incarnation of the application from the supplier hits a brick wall, and
the supplier needs to clean up its act.

OK, a 75-million is no game for kids, and it requires more careful coding
and design than a 750000 row table.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx|||Erland is correct; I personally am running an app that generates
40,000,000 rows of data a day, and it performs well (and it's not a
super server, either). I also suck data out of app by a vendor that
has less than 100,000 rows per day, and it takes hours using their
method.

Not saying you should point the finger at your vendor, but if the
performance of their app is mission-critical, and they value your
business, you need to have a conversation with them about SQL tuning.
Could be something you're doing, or it could be something they're
doing. But it's probably not the engine (unless you're running SQL
Server on a bare-bones server).

Stu|||chloe.crowder@.bl.uk wrote:
> Hi
> We received the following in an email from a third-party supplier (who
> naturally has a solution for the problem as described). It sounds like
> gibberish to me, but does anyone have any comments?
> <quote>
> SQL in its current incarnation hits a performance brick wall when a
> table contains more than about 75 million rows. This is not a
> configuration limit as the table could be grown a lot larger but the
> performance issue generates problems for ??; primarily during
> search and retrieval of archived objects; although if the database
> engine is being heavily hit for retrieval the archiving process can
> slow down as well.
> </quote
Translation 1:

We charge little money for our services, and we need to have profits so
we contract amateurs to do the job and when there are problems we
complain against something/someone not related to us. If you want a
solution pay us more money.

Translation 2:

We didn't think about a table with 75 million rows, we don't want to
look like stupids and lose you as a customer.

etc, etc...

One thing is sure, your supplier is not honest, don't trust them!

> Chloe Crowder
> British Library|||Thanks everyone - I was 99.9% sure it was rubbish and the problem was
with their app, but I thought I should check.

Chloe

Friday, February 24, 2012

limit the resources of a job

hi
we develop a stored procedure that run a dll, the dll consumes all the
processor, and take a long of time (2 days) processing data.
its posible limit the amount of cpu uses by a single stored procedure?
i know that using the windows system resource manager its posible limit
the amount of cpu uses by all sql server, but it's possible for only a
stored procedure?
thanksSQL Server can't control what your external process does. You said it
is the DLL and not the proc that consumes your resources.
Two days is an extremely long time to execute a proc. Why not invoke
your code from outside SQL Server using .NET or VB or something else?
That way you may be able to add some code to monitor and control what
happens during processing.
David Portas, SQL Server MVP
Whenever possible please post enough code to reproduce your problem.
Including CREATE TABLE and INSERT statements usually helps.
State what version of SQL Server you are using and specify the content
of any error messages.
SQL Server Books Online:
http://msdn2.microsoft.com/library/ms130214(en-US,SQL.90).aspx
--|||Alternatively, you may try to relax the grip on the processor (e.g. by not
using tight loops) in the code of the DLL if that doesn't lead to
unacceptable performance.
Linchi
"hongo32" wrote:

> hi
> we develop a stored procedure that run a dll, the dll consumes all the
> processor, and take a long of time (2 days) processing data.
> its posible limit the amount of cpu uses by a single stored procedure?
> i know that using the windows system resource manager its posible limit
> the amount of cpu uses by all sql server, but it's possible for only a
> stored procedure?
> thanks
>

limit the resources of a job

hi
we develop a stored procedure that run a dll, the dll consumes all the
processor, and take a long of time (2 days) processing data.
its posible limit the amount of cpu uses by a single stored procedure?
i know that using the windows system resource manager its posible limit
the amount of cpu uses by all sql server, but it's possible for only a
stored procedure?
thanks
SQL Server can't control what your external process does. You said it
is the DLL and not the proc that consumes your resources.
Two days is an extremely long time to execute a proc. Why not invoke
your code from outside SQL Server using .NET or VB or something else?
That way you may be able to add some code to monitor and control what
happens during processing.
David Portas, SQL Server MVP
Whenever possible please post enough code to reproduce your problem.
Including CREATE TABLE and INSERT statements usually helps.
State what version of SQL Server you are using and specify the content
of any error messages.
SQL Server Books Online:
http://msdn2.microsoft.com/library/ms130214(en-US,SQL.90).aspx
|||Alternatively, you may try to relax the grip on the processor (e.g. by not
using tight loops) in the code of the DLL if that doesn't lead to
unacceptable performance.
Linchi
"hongo32" wrote:

> hi
> we develop a stored procedure that run a dll, the dll consumes all the
> processor, and take a long of time (2 days) processing data.
> its posible limit the amount of cpu uses by a single stored procedure?
> i know that using the windows system resource manager its posible limit
> the amount of cpu uses by all sql server, but it's possible for only a
> stored procedure?
> thanks
>