c# - I am using large amount of data for comparison.. is there any idea for paging the data to datatable -

i using large amount of data comparison, while fetching records db getting hundreds of thousands of records , inserting datatable, using more memory , performance very slow.. there idea paging data datatable.

there way it. may 1 of many solutions. if lets say, have table maintable, create temp table temptable follows

select id, row_number() over(order id) rownum temptable  maintable 

this create reference records @ moment of job start. so, if records deleted or added, not break code.

now, in c#, first, retrieve max(rownum) , using , pagesize write logic, in select portions of records in calculated loop retrieve data following

select mt.* maintable mt inner join temptable tt on mt.id = tt.id , tt.rownum between <startrow> , <endrow> 

where startrow , endrow calculated values. return of such query may not page size setup, if rows deleted while long-running job running. , clean temp table.